-- Logs begin at Sun 2022-09-04 00:10:31 UTC, end at Sun 2022-09-04 01:26:55 UTC. -- Sep 04 00:56:05.419471 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Sep 04 00:56:05.428194 dv-cinder-master-3-iscsi devstack@n-api.service[106006]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Sep 04 00:56:05.430160 dv-cinder-master-3-iscsi devstack@n-api.service[106006]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Sep 4 00:56:05 2022] *** Sep 04 00:56:05.430160 dv-cinder-master-3-iscsi devstack@n-api.service[106006]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 04 00:56:05.430160 dv-cinder-master-3-iscsi devstack@n-api.service[106006]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 04 00:56:05.430160 dv-cinder-master-3-iscsi devstack@n-api.service[106006]: nodename: dv-cinder-master-3-iscsi Sep 04 00:56:05.430160 dv-cinder-master-3-iscsi devstack@n-api.service[106006]: machine: x86_64 Sep 04 00:56:05.430160 dv-cinder-master-3-iscsi devstack@n-api.service[106006]: clock source: unix Sep 04 00:56:05.430160 dv-cinder-master-3-iscsi devstack@n-api.service[106006]: pcre jit disabled Sep 04 00:56:05.430160 dv-cinder-master-3-iscsi devstack@n-api.service[106006]: detected number of CPU cores: 4 Sep 04 00:56:05.430160 dv-cinder-master-3-iscsi devstack@n-api.service[106006]: current working directory: / Sep 04 00:56:05.430160 dv-cinder-master-3-iscsi devstack@n-api.service[106006]: detected binary path: /usr/bin/uwsgi-core Sep 04 00:56:05.430771 dv-cinder-master-3-iscsi devstack@n-api.service[106006]: your processes number limit is 47680 Sep 04 00:56:05.430771 dv-cinder-master-3-iscsi devstack@n-api.service[106006]: your memory page size is 4096 bytes Sep 04 00:56:05.430771 dv-cinder-master-3-iscsi devstack@n-api.service[106006]: detected max file descriptor number: 2048 Sep 04 00:56:05.430771 dv-cinder-master-3-iscsi devstack@n-api.service[106006]: lock engine: pthread robust mutexes Sep 04 00:56:05.430771 dv-cinder-master-3-iscsi devstack@n-api.service[106006]: thunder lock: enabled Sep 04 00:56:05.430771 dv-cinder-master-3-iscsi devstack@n-api.service[106006]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Sep 04 00:56:05.430771 dv-cinder-master-3-iscsi devstack@n-api.service[106006]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 04 00:56:05.469155 dv-cinder-master-3-iscsi devstack@n-api.service[106006]: Python main interpreter initialized at 0x55ade12779c0 Sep 04 00:56:05.469155 dv-cinder-master-3-iscsi devstack@n-api.service[106006]: python threads support enabled Sep 04 00:56:05.469155 dv-cinder-master-3-iscsi devstack@n-api.service[106006]: your server socket listen backlog is limited to 100 connections Sep 04 00:56:05.469155 dv-cinder-master-3-iscsi devstack@n-api.service[106006]: your mercy for graceful operations on workers is 90 seconds Sep 04 00:56:05.469503 dv-cinder-master-3-iscsi devstack@n-api.service[106006]: mapped 403077 bytes (393 KB) for 2 cores Sep 04 00:56:05.469588 dv-cinder-master-3-iscsi devstack@n-api.service[106006]: *** Operational MODE: preforking *** Sep 04 00:56:05.469834 dv-cinder-master-3-iscsi devstack@n-api.service[106006]: *** uWSGI is running in multiple interpreter mode *** Sep 04 00:56:05.469834 dv-cinder-master-3-iscsi devstack@n-api.service[106006]: spawned uWSGI master process (pid: 106006) Sep 04 00:56:05.470020 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Sep 04 00:56:05.471100 dv-cinder-master-3-iscsi devstack@n-api.service[106006]: spawned uWSGI worker 1 (pid: 106007, cores: 1) Sep 04 00:56:05.471100 dv-cinder-master-3-iscsi devstack@n-api.service[106006]: spawned uWSGI worker 2 (pid: 106008, cores: 1) Sep 04 00:56:05.471100 dv-cinder-master-3-iscsi devstack@n-api.service[106006]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 04 00:56:06.144721 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 04 00:56:06.173533 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 04 00:56:07.249747 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 04 00:56:07.249747 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: from cryptography.utils import int_from_bytes Sep 04 00:56:07.250354 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 04 00:56:07.250354 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: from cryptography.utils import int_from_bytes Sep 04 00:56:07.250539 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 04 00:56:07.250539 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: from cryptography.utils import int_from_bytes Sep 04 00:56:07.250946 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 04 00:56:07.250946 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: from cryptography.utils import int_from_bytes Sep 04 00:56:09.745808 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 04 00:56:09.746338 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 04 00:56:09.746498 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 04 00:56:09.746857 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 04 00:56:09.747113 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 04 00:56:09.747497 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.747819 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.748143 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.748253 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 04 00:56:09.748629 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 04 00:56:09.748723 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.748816 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.748978 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 04 00:56:09.749109 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.749337 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 04 00:56:09.749435 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.749591 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 04 00:56:09.749690 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.749978 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.750075 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.750233 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.750326 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.750542 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.750637 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.750861 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.750949 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.751177 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.751264 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.751477 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.751571 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.751782 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.751868 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.752095 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.752195 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.752397 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.752502 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.752692 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.752787 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.752997 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.753115 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.753354 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.753518 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.753678 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.753962 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.754075 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.754266 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.754369 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.754575 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.754667 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.754878 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.754987 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.755198 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.755287 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.755502 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.755588 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.756003 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.756187 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.756293 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.756499 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.756606 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.756820 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.756907 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.757165 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.757280 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.757548 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.757645 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.757857 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.757954 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.758169 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.758277 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.758473 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.758567 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.758799 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.758958 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.759096 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.759182 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.759468 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.759566 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.759784 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.759893 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.760094 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.760181 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.760428 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.760591 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.760835 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.760941 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.761165 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.761268 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.761477 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.761565 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.761797 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.761974 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.762095 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.762190 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.762412 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.762512 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.762704 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.762791 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.763017 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.763117 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.763318 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.763412 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.763618 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.763717 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.763864 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.763951 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.764167 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.764254 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.764485 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.764576 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.764803 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.764889 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.765142 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.765249 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.765394 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.765482 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.765641 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.765746 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.765956 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.766062 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.766271 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.766364 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.766576 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.766666 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.766879 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.766975 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.767139 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.767224 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.767392 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.767498 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.767713 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.767821 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.768030 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.768117 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.768341 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.768447 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.768638 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.768730 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.768942 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.769026 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.769271 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.769368 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.769578 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.769665 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.769879 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.769963 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.770183 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.770277 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.770488 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.770583 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.42 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.770806 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.42 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.770898 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.771109 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.771219 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.771428 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.771513 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.771734 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.771821 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.772036 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.772122 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.772351 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.772437 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.42 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.772668 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.772754 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.42 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.772972 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.773058 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.773302 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.773389 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.773604 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.773698 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.773920 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.774015 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.774237 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.774322 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.774539 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.774625 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.774786 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.774874 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.775080 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.775175 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.775380 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.775474 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.775691 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.775778 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.775992 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.776635 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.776832 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.776940 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.777131 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.777283 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.777431 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.777528 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.777781 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.777890 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.778038 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.778124 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.778343 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.778472 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.778647 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.778757 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.778945 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.779049 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.779256 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.779349 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.779534 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.779638 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.779835 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.779947 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.780137 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.780269 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.780435 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.780526 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.780737 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.780840 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.781043 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.781152 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.781368 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.781459 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.781669 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.781771 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.781971 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.782058 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.782272 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.782359 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.782585 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.782670 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.782888 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.782976 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.783190 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.783303 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.783492 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.783604 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.783792 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.783879 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.784094 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.784181 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.784406 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.784505 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.784707 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.784794 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.785007 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.785128 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.785347 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.785438 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.785642 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.785737 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.785943 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.786053 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.786253 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.786341 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.786556 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.786645 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.786857 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.786944 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.787153 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.787238 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.787455 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.787559 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.787759 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.787846 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.788075 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.788171 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.788379 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.788545 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.788677 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.788777 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.788971 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.789060 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.789300 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.789398 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.789608 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.789704 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.789907 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.790000 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.790206 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.790303 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.790506 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.790601 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.790805 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.790900 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.791123 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.791220 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.791445 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.791540 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.792296 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.792472 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.792613 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.792792 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.792921 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.793146 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.793252 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 00:56:09.793480 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.793583 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.793799 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.793909 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.794110 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.794255 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.794423 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.794573 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.794748 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.794907 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.795060 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.795239 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.795371 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.795561 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.795681 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.795873 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.795997 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.796194 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.796326 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.796520 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.796610 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.796835 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.796943 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.797169 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.797275 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.797485 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.797581 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.797806 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.797893 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.798129 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.798243 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.798457 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.798560 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.798775 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.798871 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.799083 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.799180 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.799401 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.799507 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.799715 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.799806 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.800032 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.800128 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.800353 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.800440 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.800672 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.800760 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.800981 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.801067 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.801324 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.801423 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.801635 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.801730 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.801979 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.802067 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.802291 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.802385 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.802601 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.802688 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.802916 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.803008 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.803224 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.803309 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.803536 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.803628 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.803845 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.803931 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.804155 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.804242 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.804465 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.804567 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.804774 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.804861 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.805102 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.805191 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.805430 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.805530 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.805740 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.805825 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.806051 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.806136 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.806361 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.806459 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.806670 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.806765 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.806983 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.807069 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.807339 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.807590 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.807772 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.807915 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.808199 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.808412 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.808505 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.808604 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.808777 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.808888 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.809106 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.809206 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.809412 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.809503 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.809717 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.809829 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.810032 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.810123 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.810359 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.810449 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.810663 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.810760 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.810969 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.811071 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.811280 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.811368 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.811583 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.811671 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.811892 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.811980 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.812201 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.812294 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.812504 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.812592 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.812813 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.812912 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.813152 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.813266 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.813457 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.813553 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.813772 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.813873 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.814081 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.814168 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.814384 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.814470 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.814700 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.814797 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.815000 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.815087 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.815303 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.815391 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.815615 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.815716 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.815928 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.816023 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.816233 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.816472 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.816570 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.816653 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.816841 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.816931 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.817165 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.817256 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.817483 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.817573 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.817788 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.817875 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.818099 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.818186 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.818414 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.818512 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.818725 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.818811 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.819046 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.819141 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.819359 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.819448 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.819666 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.819770 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.819969 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.820078 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.820274 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.820363 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.820573 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.820662 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.820870 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.820971 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.821192 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.821298 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.821500 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.821596 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.821798 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.821894 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.822096 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.822185 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.822394 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.822483 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.822691 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.822787 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.823030 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.823126 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.823339 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.823471 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.823646 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.823737 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.823950 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.824046 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.824254 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.824351 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.824558 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.824655 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.824860 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.824956 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.825202 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.825293 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.825516 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.825605 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.825819 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.825909 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.826121 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.826216 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.826423 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.826527 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.826757 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.826852 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.827060 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.827157 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.827367 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.827454 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.827676 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.827760 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.827986 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.828083 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.828294 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.828387 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.828610 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.828699 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.828918 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.829019 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.829261 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.829359 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.829568 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.829668 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.829889 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.830003 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.830199 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.830299 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.830512 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.830602 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.830811 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.830907 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.831123 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.831229 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.831432 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.831520 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.831757 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.831847 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.832072 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.832159 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.832391 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.832478 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.832693 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.832781 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.833004 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.833144 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.833339 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.833427 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.833647 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.833738 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.833954 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.834042 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.834260 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.834355 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.834575 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.834664 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.834880 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.834976 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.835193 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.835291 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.835500 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.835595 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.835810 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.835897 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.836113 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.836208 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.836428 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.836516 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.836728 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.836832 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.837036 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.837158 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.837373 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.837462 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.837696 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.837787 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.838007 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.838106 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.838313 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.838400 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.839151 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.839303 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.839462 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.839562 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.839769 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.839931 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.840077 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.840278 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.840407 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.840594 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.840693 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.840909 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.841007 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.841258 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.841435 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.841569 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.841666 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.841875 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.841961 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.842175 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.842271 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.842479 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.842572 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.842780 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.842872 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.843081 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.843170 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.843381 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.843474 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.843685 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.843780 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.843994 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.844091 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.844298 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.844392 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.844613 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.844699 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.844922 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.845010 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.845247 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.845336 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.845550 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.845637 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.845863 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.845957 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.846169 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.846254 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.846471 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.846568 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.846773 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.846872 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.847073 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.847168 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.847394 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.847500 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.847699 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.847785 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.847995 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.848083 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.848311 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.848407 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.848617 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.848722 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.848920 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.849007 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.849254 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.849425 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.849569 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.849703 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.849837 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.849979 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.850154 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.850322 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.850457 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.850606 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.850741 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.850874 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.851049 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.851187 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.851355 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.851493 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.851664 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.851801 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.851971 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.852109 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.852289 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.852434 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.852588 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.852724 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.852890 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.853034 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.853220 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.853312 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.853522 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.853617 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.853827 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.853917 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.854731 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.854856 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.855205 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.855440 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.855538 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.855781 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.855877 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.856102 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.856190 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.856415 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.856509 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.856730 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.856818 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.857053 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.857196 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.857409 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.857635 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.857733 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.857974 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.858063 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.858295 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.858383 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.858620 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.858706 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.858920 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.859006 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.859223 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.859320 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.859535 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.859641 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.859836 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.859925 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.860137 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.860224 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.860437 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.860536 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.860744 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.860839 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.861043 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.861169 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.861367 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.861468 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.861717 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.861803 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.862026 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.862115 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.862333 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.862422 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.862631 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.862718 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.862944 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.863039 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.863246 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.863335 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.863547 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.863642 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.863851 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.863939 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.864149 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.864237 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.864448 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.864543 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.864748 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.864835 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.865057 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.865175 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.865381 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.865470 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.865703 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.865878 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.866014 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.866103 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.866317 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.866406 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.866620 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.866721 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.866923 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.867011 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.867243 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.867330 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.867547 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.867645 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.867848 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.868186 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.868402 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.868491 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.868716 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.868801 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.869020 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.869141 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.869347 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.869435 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.869656 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.869772 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.870069 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.870156 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.870389 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.870485 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.870667 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.870773 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.870979 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.871067 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.871290 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.871379 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.871587 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.871681 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.871896 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.871982 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.872198 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.872296 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.872498 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.872595 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.872798 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.872886 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.873125 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.873231 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.873433 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.873522 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.873735 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.873829 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.874042 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.874137 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.874342 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.874431 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.874642 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.874736 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.874941 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.875044 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.875240 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.875329 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.875539 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.875627 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.875838 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.875926 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.876143 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.876237 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.876441 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.876530 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.876752 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.876841 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.877057 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.877159 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.877381 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.877470 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.877681 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.877780 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.877989 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.878075 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.878287 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.878372 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.878589 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.878685 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.878902 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.878990 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.879212 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.879311 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.879517 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.879604 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.879816 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.879912 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.880125 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.880221 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.880427 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.880513 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.880724 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.880816 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.881023 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.881144 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.881358 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.881447 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.881658 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.881746 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.881959 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.882057 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.882259 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.882348 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.882558 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.882646 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.882858 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.882947 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.883160 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.883261 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.883474 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.883579 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.883775 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.883863 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.884076 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.884162 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.884376 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.884462 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.884677 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.884773 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.884980 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.885069 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.885329 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.885456 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.885695 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.885861 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.886090 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.886186 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.886403 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.886499 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.886714 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.886800 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.887024 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.887112 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.887337 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.887446 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.887638 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.887727 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.887949 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.888044 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.888258 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.888356 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.888568 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.888656 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.888875 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.888972 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.889205 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.889318 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.889519 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.889614 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.889827 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.889932 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.890135 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.890232 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.890447 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.890541 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.890761 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.890850 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.891065 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.891155 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.891367 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.891453 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.891677 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.891765 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.891985 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.892074 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.892294 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.892383 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.892598 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.892692 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.892908 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.893005 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.893240 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.893336 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.893552 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.893652 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.893861 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.893957 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.894169 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.894258 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.894476 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.894642 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.894779 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.894876 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.895087 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.895177 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.895394 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.895491 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.895695 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.895790 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.896004 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.896090 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.896308 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.896412 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.896610 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.896711 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.896914 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.897001 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.897237 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.897339 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.897547 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.897642 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.897847 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.897933 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.898169 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.898256 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.898481 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.898568 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.898785 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.898873 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.899096 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.899185 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.899403 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.899490 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.899714 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.899803 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.900020 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.900117 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.900334 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.900445 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.900634 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.900760 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.901588 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.901739 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.901899 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.902002 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.902209 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.902307 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.902517 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.902606 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.902819 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.902921 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.903121 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.903218 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.903429 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.903526 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.903748 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.903889 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.904050 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.904204 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.904352 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.904518 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.904652 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.904830 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.904971 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.905161 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.905294 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.905467 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.905612 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.905779 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.906022 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.906119 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.906330 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.906419 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.906630 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.906718 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.906974 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.907069 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.907297 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.907440 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.907601 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.907697 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.907921 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.908010 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.908236 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.908323 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.908556 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.908645 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.908862 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.908960 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.909195 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.909358 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.909504 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.909672 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.909825 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.909985 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.910144 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.910307 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.910453 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.910630 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.910765 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.910953 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.911096 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.911260 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.911370 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.911573 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.911686 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.911883 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.911995 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.912206 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.912315 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.912532 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.912721 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.912850 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.913040 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.913189 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.913373 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.913495 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.913689 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.913817 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.914013 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.914137 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.914318 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.914447 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.914620 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.914768 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.914931 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.915179 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.915267 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.915502 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.915603 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.915813 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.915910 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.916114 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.916201 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.916440 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.916580 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.917325 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.917443 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.917641 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.917836 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.917959 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.918146 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.918268 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.918451 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.918577 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.918783 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.918901 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.919088 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.919207 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.919413 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.919511 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.919725 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.919823 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.920047 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.920136 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.920367 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.920456 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.920671 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.920776 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.920989 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.921107 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.921317 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.921413 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.921622 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.921718 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.921925 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.922014 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.922227 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.922315 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.922559 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.922654 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.922857 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.922951 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.923175 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.923263 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.923482 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.923571 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.923789 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.923885 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.924093 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.924185 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.924399 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.924488 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.924728 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.924816 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.925031 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.925174 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.925366 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.925456 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.925678 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.925767 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.925999 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.926101 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.926300 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.926389 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.926598 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.926689 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.926910 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.927044 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.927208 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.927300 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.927509 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.927613 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.927806 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.927912 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.928112 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.928200 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.928410 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.928497 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.928714 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.928817 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.929010 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.929142 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.929347 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.929447 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.929644 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.929738 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.929945 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.930039 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.930256 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.930354 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.930554 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.930641 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.930850 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.930935 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.931147 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.931252 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.931468 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.931562 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.931768 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.931854 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.932089 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.932183 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.932438 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.932527 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.932820 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.932906 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.933154 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.933260 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.933464 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.933559 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.933775 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.933861 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.934087 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.934187 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.934400 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.934488 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.934713 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.934799 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.935024 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.935167 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.935332 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.935421 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.935645 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.935732 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.935960 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.936055 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.936273 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.936366 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.936590 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.936782 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.936906 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.937125 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.937248 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.937441 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.937561 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.937756 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.937882 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.938066 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.938167 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.938376 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.938487 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.938691 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.939001 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.939148 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.939311 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.939402 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.939621 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.939731 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.939926 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.940016 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.940245 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.940385 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.940559 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.940656 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.940874 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.940961 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.941213 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.941321 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.941531 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.941643 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.941840 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.941930 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.942150 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.942241 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.942461 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.942552 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.942789 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.942877 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.943110 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.943198 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.943415 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.943513 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.943724 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.943818 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.944036 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.944135 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.944346 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.944443 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.944656 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.944745 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.944962 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.945058 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.945298 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.945388 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.945609 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.945697 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.945919 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.946023 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.946227 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.946324 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.946537 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.946634 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.946847 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.946940 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.947157 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.947248 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.947473 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.947561 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.947791 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.947925 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.948148 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.948328 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.948464 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.948777 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.948883 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.949107 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.949377 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.949474 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.949761 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.949901 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.950083 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.950222 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.950398 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.950558 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.950714 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.950886 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.951030 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.951219 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.951540 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.951644 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.951856 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.951963 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.952180 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.952276 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.952485 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.952590 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.952790 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.952897 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.953126 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.953236 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.953440 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.953537 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.953742 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.953838 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.954062 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.954164 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.954370 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.954466 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.954670 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.954765 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.954984 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.955086 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.955290 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.955386 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.955614 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.955814 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.955910 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.956159 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.956255 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.956472 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.956580 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.956793 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.956897 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.957134 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.957363 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.957469 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.957711 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.957808 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.958026 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.958122 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.958342 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.958445 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.958650 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.958753 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.958961 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.959067 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.959267 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.959372 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.959565 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.959667 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.959864 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.959960 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.960174 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.960278 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.960481 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.960578 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.960780 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.960875 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.961106 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.961204 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.961403 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.961493 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.961714 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.961802 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.962015 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.962101 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.962314 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.962401 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.962632 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.962720 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.962939 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.963036 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.963265 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.963360 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.964238 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.964428 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.964552 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.964750 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.964859 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.965059 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.965207 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.965398 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.965517 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.965706 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.965822 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.966020 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.966116 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.966326 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.966422 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.966637 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.966725 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.966982 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.967077 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.967293 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.967389 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.967595 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.967690 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.967906 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.967992 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.968208 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.968303 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.968510 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.968606 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.968826 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.968927 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.969161 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.969269 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.969469 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.969556 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.969774 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.969873 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.970086 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.970190 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.970396 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.970489 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.970719 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.970816 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.971034 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.971248 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.971344 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.971582 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.971676 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.971902 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.971992 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.972239 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.972506 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.972594 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.972842 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.972931 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.973184 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.973284 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.973493 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.973581 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.973819 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.973915 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.974134 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.974229 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.974439 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.974538 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.974747 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.974836 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.975055 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.975143 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.975364 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.975465 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.975676 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.975764 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.975983 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.976077 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.976295 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.976390 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.976603 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.976690 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.976914 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.977000 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.977334 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.977481 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.977610 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.977697 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.977842 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.977939 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.978152 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.978241 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.978463 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.978561 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.978773 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.978868 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.979087 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.979726 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.979956 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.980214 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.980306 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.980387 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.980577 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.980707 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.980887 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.981031 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.981222 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.981379 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.981531 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.981702 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.981839 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.982021 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.982155 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.982334 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.982485 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.982645 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.982802 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.982957 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.983106 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.983274 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.983594 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.983766 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.983927 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.984081 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.984243 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.984400 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.984563 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.984721 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.985036 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.985254 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.985381 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.985578 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.985704 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.985906 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.986017 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.986235 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.986333 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.986556 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.986654 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.986881 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.986981 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.987209 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.987307 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.987526 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.987615 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.987853 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.987942 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.988180 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.988276 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.988482 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.988578 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.988786 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.988883 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.989120 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.989280 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.989441 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.989575 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.989757 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.989895 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.990061 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.990195 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.990362 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.990505 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.990673 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.990816 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.990975 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.991110 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.991277 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.991422 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.991579 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.991724 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.991880 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.992014 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.992190 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.992326 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.992491 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.992637 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.992777 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.993188 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.993358 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.993499 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.993663 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.993810 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.993945 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.994075 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.994250 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.994394 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.994540 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.994686 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.994863 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.995123 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.995313 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.995463 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.995673 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.995851 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.996002 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.996136 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.996274 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.996363 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.996452 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.996598 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.996687 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.996766 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.996917 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.997014 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.997247 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.997339 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.997554 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.997641 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.997886 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.997984 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.998222 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.998324 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.998546 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.998646 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.998860 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.998960 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.999172 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.999258 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.999481 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.999566 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.999799 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:09.999893 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.000115 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.000249 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.000426 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.000523 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.000737 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.000838 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.001049 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.001178 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.001384 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.001473 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.001697 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.001787 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.002008 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.002101 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.002323 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.002410 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.002624 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.002711 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.002937 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.003023 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.003249 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.003335 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.003565 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.003655 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.003881 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.003977 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.004184 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.004278 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.004508 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.004601 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.004818 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.004906 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.005155 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.005305 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.005467 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.005553 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.005775 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.005861 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.006080 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.006176 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.006394 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.006482 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.006702 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.006788 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.007010 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.007097 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.007323 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.007408 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.007633 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.007728 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.007953 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.008043 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.008264 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.008353 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.008576 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.008699 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.008890 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.008988 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.009229 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.009344 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.009557 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.009668 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.009876 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.009973 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.010185 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.010294 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.010538 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.010694 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.010893 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.010988 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.011206 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.011295 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.011510 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.011610 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.011751 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=106007) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 04 00:56:10.011906 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.012237 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.012553 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.012865 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.013196 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 00:56:10.013440 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=106008) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 04 00:56:10.165815 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO dbcounter [None req-4757fb2b-cc94-44a1-a7c2-d938866a06e2 None None] Registered counter for database nova_api Sep 04 00:56:10.166656 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO dbcounter [None req-346211b5-bca5-4ec9-9585-3e4fa3a883f6 None None] Registered counter for database nova_api Sep 04 00:56:10.187965 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG oslo_db.sqlalchemy.engines [None req-4757fb2b-cc94-44a1-a7c2-d938866a06e2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=106007) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 04 00:56:10.189762 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG oslo_db.sqlalchemy.engines [None req-346211b5-bca5-4ec9-9585-3e4fa3a883f6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=106008) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 04 00:56:10.194574 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG dbcounter [-] [106007] Writer thread running {{(pid=106007) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 04 00:56:10.196178 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG dbcounter [-] [106008] Writer thread running {{(pid=106008) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 04 00:56:10.252946 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.context [None req-346211b5-bca5-4ec9-9585-3e4fa3a883f6 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),cf96709b-43a7-4964-94cf-103a5b1a1016(cell1) {{(pid=106008) load_cells /opt/stack/nova/nova/context.py:464}} Sep 04 00:56:10.253147 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.context [None req-4757fb2b-cc94-44a1-a7c2-d938866a06e2 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),cf96709b-43a7-4964-94cf-103a5b1a1016(cell1) {{(pid=106007) load_cells /opt/stack/nova/nova/context.py:464}} Sep 04 00:56:10.256725 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG oslo_concurrency.lockutils [None req-346211b5-bca5-4ec9-9585-3e4fa3a883f6 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=106008) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 00:56:10.256852 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG oslo_concurrency.lockutils [None req-4757fb2b-cc94-44a1-a7c2-d938866a06e2 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=106007) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 00:56:10.257233 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG oslo_concurrency.lockutils [None req-346211b5-bca5-4ec9-9585-3e4fa3a883f6 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=106008) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 00:56:10.257328 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG oslo_concurrency.lockutils [None req-4757fb2b-cc94-44a1-a7c2-d938866a06e2 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=106007) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 00:56:10.258787 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG oslo_concurrency.lockutils [None req-346211b5-bca5-4ec9-9585-3e4fa3a883f6 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=106008) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 00:56:10.258904 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG oslo_concurrency.lockutils [None req-4757fb2b-cc94-44a1-a7c2-d938866a06e2 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=106007) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 00:56:10.259502 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG oslo_concurrency.lockutils [None req-346211b5-bca5-4ec9-9585-3e4fa3a883f6 None None] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=106008) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 00:56:10.259594 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG oslo_concurrency.lockutils [None req-4757fb2b-cc94-44a1-a7c2-d938866a06e2 None None] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=106007) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 00:56:10.259942 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG oslo_concurrency.lockutils [None req-346211b5-bca5-4ec9-9585-3e4fa3a883f6 None None] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=106008) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 00:56:10.260080 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG oslo_concurrency.lockutils [None req-4757fb2b-cc94-44a1-a7c2-d938866a06e2 None None] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=106007) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 00:56:10.261971 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG oslo_concurrency.lockutils [None req-4757fb2b-cc94-44a1-a7c2-d938866a06e2 None None] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=106007) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 00:56:10.262119 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG oslo_concurrency.lockutils [None req-346211b5-bca5-4ec9-9585-3e4fa3a883f6 None None] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=106008) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 00:56:10.263941 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO dbcounter [None req-4757fb2b-cc94-44a1-a7c2-d938866a06e2 None None] Registered counter for database nova_cell0 Sep 04 00:56:10.264094 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO dbcounter [None req-346211b5-bca5-4ec9-9585-3e4fa3a883f6 None None] Registered counter for database nova_cell0 Sep 04 00:56:10.268158 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO dbcounter [None req-346211b5-bca5-4ec9-9585-3e4fa3a883f6 None None] Registered counter for database nova_cell1 Sep 04 00:56:10.268281 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO dbcounter [None req-4757fb2b-cc94-44a1-a7c2-d938866a06e2 None None] Registered counter for database nova_cell1 Sep 04 00:56:10.275752 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG oslo_db.sqlalchemy.engines [None req-346211b5-bca5-4ec9-9585-3e4fa3a883f6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=106008) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 04 00:56:10.277476 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG oslo_db.sqlalchemy.engines [None req-4757fb2b-cc94-44a1-a7c2-d938866a06e2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=106007) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 04 00:56:10.280067 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG oslo_db.sqlalchemy.engines [None req-4757fb2b-cc94-44a1-a7c2-d938866a06e2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=106007) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 04 00:56:10.280654 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG dbcounter [-] [106008] Writer thread running {{(pid=106008) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 04 00:56:10.281336 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG oslo_db.sqlalchemy.engines [None req-346211b5-bca5-4ec9-9585-3e4fa3a883f6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=106008) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 04 00:56:10.283929 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG dbcounter [-] [106007] Writer thread running {{(pid=106007) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 04 00:56:10.420067 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG dbcounter [-] [106008] Writer thread running {{(pid=106008) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 04 00:56:10.421260 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG dbcounter [-] [106007] Writer thread running {{(pid=106007) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 04 00:56:10.430372 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO dbcounter [None req-346211b5-bca5-4ec9-9585-3e4fa3a883f6 None None] Registered counter for database nova_cell0 Sep 04 00:56:10.431033 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO dbcounter [None req-4757fb2b-cc94-44a1-a7c2-d938866a06e2 None None] Registered counter for database nova_cell0 Sep 04 00:56:10.437841 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG oslo_db.sqlalchemy.engines [None req-346211b5-bca5-4ec9-9585-3e4fa3a883f6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=106008) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 04 00:56:10.439341 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG oslo_db.sqlalchemy.engines [None req-4757fb2b-cc94-44a1-a7c2-d938866a06e2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=106007) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 04 00:56:10.442692 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG dbcounter [-] [106008] Writer thread running {{(pid=106008) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 04 00:56:10.443999 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG dbcounter [-] [106007] Writer thread running {{(pid=106007) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 04 00:56:10.899877 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: WARNING keystonemiddleware.auth_token [None req-4757fb2b-cc94-44a1-a7c2-d938866a06e2 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 04 00:56:10.907283 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: WARNING keystonemiddleware.auth_token [None req-4757fb2b-cc94-44a1-a7c2-d938866a06e2 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 04 00:56:10.930530 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: WARNING keystonemiddleware.auth_token [None req-346211b5-bca5-4ec9-9585-3e4fa3a883f6 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 04 00:56:10.938403 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: WARNING keystonemiddleware.auth_token [None req-346211b5-bca5-4ec9-9585-3e4fa3a883f6 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 04 00:56:11.251415 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: WARNING keystonemiddleware.auth_token [None req-4757fb2b-cc94-44a1-a7c2-d938866a06e2 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 04 00:56:11.257314 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: WARNING keystonemiddleware.auth_token [None req-4757fb2b-cc94-44a1-a7c2-d938866a06e2 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 04 00:56:11.263663 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55ade12779c0 pid: 106007 (default app) Sep 04 00:56:11.265839 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-4757fb2b-cc94-44a1-a7c2-d938866a06e2 None None] Calling method '>' {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:56:11.267033 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-4757fb2b-cc94-44a1-a7c2-d938866a06e2 None None] 10.222.0.42 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001793 Sep 04 00:56:11.267440 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 1/1] 10.222.0.42 () {50 vars in 788 bytes} [Sun Sep 4 00:56:11 2022] GET /compute/v2.1/ => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 00:56:11.286992 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: WARNING keystonemiddleware.auth_token [None req-346211b5-bca5-4ec9-9585-3e4fa3a883f6 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 04 00:56:11.293499 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: WARNING keystonemiddleware.auth_token [None req-346211b5-bca5-4ec9-9585-3e4fa3a883f6 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 04 00:56:11.298885 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55ade12779c0 pid: 106008 (default app) Sep 04 00:57:32.951861 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-4757fb2b-cc94-44a1-a7c2-d938866a06e2 None None] Calling method '>' {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:57:32.953457 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-4757fb2b-cc94-44a1-a7c2-d938866a06e2 None None] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001986 Sep 04 00:57:32.954084 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 2/2] 10.222.0.42 () {54 vars in 931 bytes} [Sun Sep 4 00:57:32 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 00:57:32.971197 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG dbcounter [-] [106008] Writing DB stats nova_api:SELECT=3 {{(pid=106008) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:57:32.976134 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG dbcounter [-] [106008] Writing DB stats nova_cell0:SELECT=3 {{(pid=106008) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:57:32.976134 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG dbcounter [-] [106008] Writing DB stats nova_cell1:SELECT=3 {{(pid=106008) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:57:32.976134 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG dbcounter [-] [106008] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=106008) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:57:33.834550 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-515cef8a-2318-4a28-8324-c832ea385d77 admin admin] Calling method '>' {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:57:33.895083 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-515cef8a-2318-4a28-8324-c832ea385d77 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 0.933103 Sep 04 00:57:33.896562 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 1/3] 10.222.0.42 () {60 vars in 1292 bytes} [Sun Sep 4 00:57:32 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 935 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 00:57:36.293140 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-4757fb2b-cc94-44a1-a7c2-d938866a06e2 None None] Calling method '>' {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:57:36.294183 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-4757fb2b-cc94-44a1-a7c2-d938866a06e2 None None] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001421 Sep 04 00:57:36.299368 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 3/4] 10.222.0.42 () {54 vars in 931 bytes} [Sun Sep 4 00:57:36 2022] GET /compute/v2.1 => generated 388 bytes in 7 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 00:57:36.725093 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-ae70e07e-a9d7-477c-8228-2bfecf5a744d admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "id": "c1", "ram": 256, "rxtx_factor": 1.0, "vcpus": 1, "swap": 0, "os-flavor-access:is_public": true, "name": "cirros256", "disk": 1}} {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 00:57:36.841331 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-ae70e07e-a9d7-477c-8228-2bfecf5a744d admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.540892 Sep 04 00:57:36.842086 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 2/5] 10.222.0.42 () {64 vars in 1283 bytes} [Sun Sep 4 00:57:36 2022] POST /compute/v2.1/flavors => generated 407 bytes in 542 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 04 00:57:36.857725 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG dbcounter [-] [106007] Writing DB stats nova_api:SELECT=3 {{(pid=106007) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:57:36.859237 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG dbcounter [-] [106007] Writing DB stats nova_cell0:SELECT=3 {{(pid=106007) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:57:36.861617 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG dbcounter [-] [106007] Writing DB stats nova_cell1:SELECT=3 {{(pid=106007) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:57:36.862806 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG dbcounter [-] [106007] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=106007) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:57:37.072966 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-0365f536-1cec-4116-a85b-2c034b1e8135 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 00:57:37.258024 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-0365f536-1cec-4116-a85b-2c034b1e8135 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.406793 Sep 04 00:57:37.258529 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 4/6] 10.222.0.42 () {64 vars in 1336 bytes} [Sun Sep 4 00:57:36 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 00:57:39.621838 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-ae70e07e-a9d7-477c-8228-2bfecf5a744d admin admin] Calling method '>' {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:57:39.622855 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-ae70e07e-a9d7-477c-8228-2bfecf5a744d admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001504 Sep 04 00:57:39.623256 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 3/7] 10.222.0.42 () {54 vars in 931 bytes} [Sun Sep 4 00:57:39 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 00:57:40.287198 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-362fc94c-b73c-4cea-ba82-26bb656d4109 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "vcpus": 1, "id": "d1", "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "disk": 5, "swap": 0, "os-flavor-access:is_public": true, "name": "ds512M"}} {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 00:57:40.415813 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-362fc94c-b73c-4cea-ba82-26bb656d4109 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.786318 Sep 04 00:57:40.417052 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 5/8] 10.222.0.42 () {64 vars in 1283 bytes} [Sun Sep 4 00:57:39 2022] POST /compute/v2.1/flavors => generated 404 bytes in 788 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 04 00:57:40.433029 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-128034e9-b716-4802-ace3-34dfad881d96 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 00:57:40.559533 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-128034e9-b716-4802-ace3-34dfad881d96 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.130317 Sep 04 00:57:40.560320 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 4/9] 10.222.0.42 () {64 vars in 1336 bytes} [Sun Sep 4 00:57:40 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 00:57:42.913255 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-362fc94c-b73c-4cea-ba82-26bb656d4109 admin admin] Calling method '>' {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:57:42.914274 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-362fc94c-b73c-4cea-ba82-26bb656d4109 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001518 Sep 04 00:57:42.914689 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 6/10] 10.222.0.42 () {54 vars in 931 bytes} [Sun Sep 4 00:57:42 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 00:57:43.308276 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-3856c3dd-ca66-401b-95f3-895411d04530 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 1, "swap": 0, "rxtx_factor": 1.0, "id": "d2", "disk": 10, "ram": 1024, "os-flavor-access:is_public": true, "name": "ds1G", "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 00:57:43.398028 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-3856c3dd-ca66-401b-95f3-895411d04530 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.477223 Sep 04 00:57:43.398510 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 5/11] 10.222.0.42 () {64 vars in 1283 bytes} [Sun Sep 4 00:57:42 2022] POST /compute/v2.1/flavors => generated 404 bytes in 478 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 04 00:57:43.410383 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-363dc050-b1f0-4678-9c62-7af53e45dacc admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 00:57:43.509818 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-363dc050-b1f0-4678-9c62-7af53e45dacc admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.103015 Sep 04 00:57:43.510410 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 7/12] 10.222.0.42 () {64 vars in 1336 bytes} [Sun Sep 4 00:57:43 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 00:57:45.940010 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-3856c3dd-ca66-401b-95f3-895411d04530 admin admin] Calling method '>' {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:57:45.941721 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-3856c3dd-ca66-401b-95f3-895411d04530 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002136 Sep 04 00:57:45.942643 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 6/13] 10.222.0.42 () {54 vars in 931 bytes} [Sun Sep 4 00:57:45 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 00:57:46.383241 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-1eb109f0-0af8-42c0-aa66-39858cb0b3da admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 2048, "id": "d3", "name": "ds2G", "vcpus": 2, "rxtx_factor": 1.0, "disk": 10, "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0}} {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 00:57:46.463159 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-1eb109f0-0af8-42c0-aa66-39858cb0b3da admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.514026 Sep 04 00:57:46.468597 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 8/14] 10.222.0.42 () {64 vars in 1283 bytes} [Sun Sep 4 00:57:45 2022] POST /compute/v2.1/flavors => generated 404 bytes in 520 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 04 00:57:46.476282 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-c20c8184-f4ad-4ed5-8d4e-ebe45384f175 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 00:57:46.572627 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-c20c8184-f4ad-4ed5-8d4e-ebe45384f175 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.100151 Sep 04 00:57:46.573357 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 7/15] 10.222.0.42 () {64 vars in 1336 bytes} [Sun Sep 4 00:57:46 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 00:57:49.119169 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-1eb109f0-0af8-42c0-aa66-39858cb0b3da admin admin] Calling method '>' {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:57:49.120239 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-1eb109f0-0af8-42c0-aa66-39858cb0b3da admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001537 Sep 04 00:57:49.120798 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 9/16] 10.222.0.42 () {54 vars in 931 bytes} [Sun Sep 4 00:57:49 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 00:57:49.537030 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-b075b6d1-0f3f-42a9-a84e-59d54ba5d6be admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "os-flavor-access:is_public": true, "ram": 4096, "vcpus": 4, "id": "d4", "disk": 20, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "name": "ds4G"}} {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 00:57:49.657351 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-b075b6d1-0f3f-42a9-a84e-59d54ba5d6be admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.526065 Sep 04 00:57:49.657351 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 8/17] 10.222.0.42 () {64 vars in 1283 bytes} [Sun Sep 4 00:57:49 2022] POST /compute/v2.1/flavors => generated 404 bytes in 530 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 04 00:57:49.671159 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-aca879ba-f642-4b62-b1f7-bba62ba82abf admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 00:57:49.756678 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-aca879ba-f642-4b62-b1f7-bba62ba82abf admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.090606 Sep 04 00:57:49.758853 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 10/18] 10.222.0.42 () {64 vars in 1336 bytes} [Sun Sep 4 00:57:49 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 00:57:52.299084 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-b075b6d1-0f3f-42a9-a84e-59d54ba5d6be admin admin] Calling method '>' {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:57:52.300019 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-b075b6d1-0f3f-42a9-a84e-59d54ba5d6be admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001472 Sep 04 00:57:52.300421 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 9/19] 10.222.0.42 () {54 vars in 931 bytes} [Sun Sep 4 00:57:52 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 00:57:52.729989 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-b0be21a0-3d1b-45f9-a314-3c3e953657cd admin admin] Calling method '>' {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:57:52.757352 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-b0be21a0-3d1b-45f9-a314-3c3e953657cd admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.450492 Sep 04 00:57:52.757902 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 11/20] 10.222.0.42 () {60 vars in 1292 bytes} [Sun Sep 4 00:57:52 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 452 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 00:57:55.217864 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-b075b6d1-0f3f-42a9-a84e-59d54ba5d6be admin admin] Calling method '>' {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:57:55.219224 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-b075b6d1-0f3f-42a9-a84e-59d54ba5d6be admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002096 Sep 04 00:57:55.219792 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 10/21] 10.222.0.42 () {54 vars in 931 bytes} [Sun Sep 4 00:57:55 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 00:57:55.633820 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-e3bdb5a8-fb0f-48e8-a489-62cea3b26cd0 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "disk": 1, "vcpus": 1, "name": "m1.tiny", "os-flavor-access:is_public": true, "swap": 0, "id": "1", "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0}} {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 00:57:55.716734 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-e3bdb5a8-fb0f-48e8-a489-62cea3b26cd0 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.490797 Sep 04 00:57:55.717327 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 12/22] 10.222.0.42 () {64 vars in 1283 bytes} [Sun Sep 4 00:57:55 2022] POST /compute/v2.1/flavors => generated 402 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 04 00:57:55.729417 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-9f8191d6-7a32-4b4a-aec2-816e510d633e admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 00:57:55.824025 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-9f8191d6-7a32-4b4a-aec2-816e510d633e admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.098212 Sep 04 00:57:55.824584 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 11/23] 10.222.0.42 () {64 vars in 1333 bytes} [Sun Sep 4 00:57:55 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 00:57:58.247598 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-e3bdb5a8-fb0f-48e8-a489-62cea3b26cd0 admin admin] Calling method '>' {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:57:58.248594 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-e3bdb5a8-fb0f-48e8-a489-62cea3b26cd0 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001411 Sep 04 00:57:58.248994 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 13/24] 10.222.0.42 () {54 vars in 931 bytes} [Sun Sep 4 00:57:58 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 00:57:58.621288 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-fef41bba-4366-4c01-b67d-419d7d7bf41e admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "disk": 20, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "m1.small", "vcpus": 1, "rxtx_factor": 1.0, "ram": 2048, "os-flavor-access:is_public": true, "id": "2"}} {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 00:57:58.699433 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-fef41bba-4366-4c01-b67d-419d7d7bf41e admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.444407 Sep 04 00:57:58.699997 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 12/25] 10.222.0.42 () {64 vars in 1283 bytes} [Sun Sep 4 00:57:58 2022] POST /compute/v2.1/flavors => generated 405 bytes in 446 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 04 00:57:58.716893 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-f18c569d-0ea4-4d0d-834b-7ee6fa5ae16c admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 00:57:58.813302 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-f18c569d-0ea4-4d0d-834b-7ee6fa5ae16c admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.103568 Sep 04 00:57:58.813902 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 14/26] 10.222.0.42 () {64 vars in 1333 bytes} [Sun Sep 4 00:57:58 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 00:58:01.273536 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-fef41bba-4366-4c01-b67d-419d7d7bf41e admin admin] Calling method '>' {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:58:01.275134 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-fef41bba-4366-4c01-b67d-419d7d7bf41e admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002160 Sep 04 00:58:01.275701 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 13/27] 10.222.0.42 () {54 vars in 931 bytes} [Sun Sep 4 00:58:01 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 00:58:01.746780 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-307e035e-5e5b-41af-bf90-d28cb95c3974 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 40, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "vcpus": 2, "id": "3", "swap": 0, "name": "m1.medium", "ram": 4096}} {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 00:58:01.823872 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-307e035e-5e5b-41af-bf90-d28cb95c3974 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.541062 Sep 04 00:58:01.824205 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 15/28] 10.222.0.42 () {64 vars in 1283 bytes} [Sun Sep 4 00:58:01 2022] POST /compute/v2.1/flavors => generated 406 bytes in 542 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 04 00:58:01.837000 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-df728330-f04a-4915-96d1-28870b3f7fb1 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 00:58:01.915852 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-df728330-f04a-4915-96d1-28870b3f7fb1 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.082448 Sep 04 00:58:01.916782 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 14/29] 10.222.0.42 () {64 vars in 1333 bytes} [Sun Sep 4 00:58:01 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 00:58:04.459466 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-307e035e-5e5b-41af-bf90-d28cb95c3974 admin admin] Calling method '>' {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:58:04.462630 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-307e035e-5e5b-41af-bf90-d28cb95c3974 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.005711 Sep 04 00:58:04.465176 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 16/30] 10.222.0.42 () {54 vars in 931 bytes} [Sun Sep 4 00:58:04 2022] GET /compute/v2.1 => generated 388 bytes in 8 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 00:58:04.975970 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-fc3b0462-e4be-4543-9c9a-08bd78e7d344 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 8192, "disk": 80, "id": "4", "os-flavor-access:is_public": true, "vcpus": 4, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "swap": 0, "name": "m1.large"}} {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 00:58:04.982492 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG dbcounter [-] [106008] Writing DB stats nova_api:SELECT=39,nova_api:INSERT=8,nova_api:SAVEPOINT=4,nova_api:RELEASE=4 {{(pid=106008) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:58:05.087980 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-fc3b0462-e4be-4543-9c9a-08bd78e7d344 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.616636 Sep 04 00:58:05.097147 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 15/31] 10.222.0.42 () {64 vars in 1283 bytes} [Sun Sep 4 00:58:04 2022] POST /compute/v2.1/flavors => generated 405 bytes in 618 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 04 00:58:05.103761 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-f9e52751-eede-4659-bc52-1b0bcf5b4d81 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 00:58:05.200617 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-f9e52751-eede-4659-bc52-1b0bcf5b4d81 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.100580 Sep 04 00:58:05.213104 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 17/32] 10.222.0.42 () {64 vars in 1333 bytes} [Sun Sep 4 00:58:05 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 00:58:07.658910 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-fc3b0462-e4be-4543-9c9a-08bd78e7d344 admin admin] Calling method '>' {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:58:07.660773 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-fc3b0462-e4be-4543-9c9a-08bd78e7d344 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002278 Sep 04 00:58:07.661217 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 16/33] 10.222.0.42 () {54 vars in 931 bytes} [Sun Sep 4 00:58:07 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 00:58:08.159066 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-7e615208-061f-4b59-9365-50d3693c14a4 admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "ram": 16384, "swap": 0, "disk": 160, "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "name": "m1.xlarge", "id": "5", "vcpus": 8}} {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 00:58:08.510519 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-7e615208-061f-4b59-9365-50d3693c14a4 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.842992 Sep 04 00:58:08.511069 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 18/34] 10.222.0.42 () {64 vars in 1283 bytes} [Sun Sep 4 00:58:07 2022] POST /compute/v2.1/flavors => generated 408 bytes in 844 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 04 00:58:08.523787 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-92f4fece-b2f5-418a-a1ca-92855ee16f46 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 00:58:08.610968 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-92f4fece-b2f5-418a-a1ca-92855ee16f46 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.090870 Sep 04 00:58:08.611873 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 17/35] 10.222.0.42 () {64 vars in 1333 bytes} [Sun Sep 4 00:58:08 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 00:58:27.187526 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-7e615208-061f-4b59-9365-50d3693c14a4 admin admin] Calling method '>' {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:58:27.188534 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-7e615208-061f-4b59-9365-50d3693c14a4 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001482 Sep 04 00:58:27.188929 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 19/36] 10.222.0.42 () {54 vars in 931 bytes} [Sun Sep 4 00:58:27 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 00:58:27.587987 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-75fc7116-86a9-4da4-a26c-ddd47d2492e7 admin admin] Calling method '>' {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:58:27.606571 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-75fc7116-86a9-4da4-a26c-ddd47d2492e7 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.411444 Sep 04 00:58:27.607105 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 18/37] 10.222.0.42 () {60 vars in 1292 bytes} [Sun Sep 4 00:58:27 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 413 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 00:58:28.681668 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-7e615208-061f-4b59-9365-50d3693c14a4 admin admin] Calling method '>' {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:58:28.682672 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-7e615208-061f-4b59-9365-50d3693c14a4 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001482 Sep 04 00:58:28.683070 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 20/38] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Sep 4 00:58:28 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 00:58:29.214592 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-2c17eab1-d888-4b61-8afe-517831ce5374 admin admin] Calling method '>' {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:58:29.220092 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.wsgi [None req-2c17eab1-d888-4b61-8afe-517831ce5374 admin admin] HTTP exception thrown: Flavor 42 could not be found. Sep 04 00:58:29.223346 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-2c17eab1-d888-4b61-8afe-517831ce5374 admin admin] Request method failure captured: Sep 04 00:58:29.223346 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Sep 04 00:58:29.223346 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: Accept: application/json Sep 04 00:58:29.223346 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: Accept-Encoding: gzip, deflate Sep 04 00:58:29.223346 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: Connection: keep-alive Sep 04 00:58:29.223346 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: Content-Length: 0 Sep 04 00:58:29.223346 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: Host: 10.222.0.42 Sep 04 00:58:29.223346 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: Openstack-Api-Version: compute 2.92 Sep 04 00:58:29.223346 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: Openstack-System-Scope: None Sep 04 00:58:29.223346 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: User-Agent: python-novaclient Sep 04 00:58:29.223346 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Auth-Token: gAAAAABjE_g0cMpRNGHbAkzwYIyweFTOYfYfEWZzPrgBnFPkCF6hMTJ3RD2vtzflBBuBH9vdb_rsBoTS3KXVWHLDz57YLNSdAWmxc1AJ1Dt7QZgpeA8AseSIBrpa2RfWyrkP7o2Fn34bL3b67pKtPXssWKq_nUwiEDXwX1GT-bWsOOt9GZPvOBE Sep 04 00:58:29.223346 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Domain-Id: None Sep 04 00:58:29.223346 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Domain-Name: None Sep 04 00:58:29.223346 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Identity-Status: Confirmed Sep 04 00:58:29.223346 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Is-Admin-Project: True Sep 04 00:58:29.223346 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Openstack-Nova-Api-Version: 2.92 Sep 04 00:58:29.223346 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Project-Domain-Id: default Sep 04 00:58:29.223346 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Project-Domain-Name: Default Sep 04 00:58:29.223346 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Project-Id: f25649add4b1426590abafbe586fc1fa Sep 04 00:58:29.223346 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Project-Name: admin Sep 04 00:58:29.223346 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Role: reader,admin,member Sep 04 00:58:29.223346 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Roles: reader,admin,member Sep 04 00:58:29.224836 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/f25649add4b1426590abafbe586fc1fa"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_f25649add4b1426590abafbe586fc1fa", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/f25649add4b1426590abafbe586fc1fa"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/f25649add4b1426590abafbe586fc1fa"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 00:58:29.224836 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Tenant: admin Sep 04 00:58:29.224836 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Tenant-Id: f25649add4b1426590abafbe586fc1fa Sep 04 00:58:29.224836 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Tenant-Name: admin Sep 04 00:58:29.224836 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-User: admin Sep 04 00:58:29.224836 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-User-Domain-Id: default Sep 04 00:58:29.224836 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-User-Domain-Name: Default Sep 04 00:58:29.224836 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-User-Id: 01782590dd294bd488a6d5c08a043ce1 Sep 04 00:58:29.224836 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-User-Name: admin Sep 04 00:58:29.224836 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: method: > Sep 04 00:58:29.224836 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: exception: Flavor 42 could not be found. Sep 04 00:58:29.224836 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: action_args: {'id': '42'} Sep 04 00:58:29.224836 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 00:58:29.224836 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 00:58:29.225989 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 04 00:58:29.225989 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 04 00:58:29.225989 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 04 00:58:29.225989 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 04 00:58:29.225989 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 04 00:58:29.225989 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 04 00:58:29.225989 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 04 00:58:29.225989 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 04 00:58:29.225989 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 04 00:58:29.225989 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 00:58:29.225989 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 04 00:58:29.225989 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 04 00:58:29.225989 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Sep 04 00:58:29.225989 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi Sep 04 00:58:29.225989 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 00:58:29.225989 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi Sep 04 00:58:29.225989 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 00:58:29.225989 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 00:58:29.225989 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 00:58:29.227567 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 00:58:29.227567 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 00:58:29.227567 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 00:58:29.227567 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 00:58:29.227567 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 04 00:58:29.227567 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 04 00:58:29.227567 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Sep 04 00:58:29.227567 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi Sep 04 00:58:29.227567 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 00:58:29.227567 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi Sep 04 00:58:29.227567 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 00:58:29.227567 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 00:58:29.227567 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 00:58:29.227567 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 00:58:29.227567 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 00:58:29.227567 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Sep 04 00:58:29.227567 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi Sep 04 00:58:29.227567 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-2c17eab1-d888-4b61-8afe-517831ce5374 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=106008) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 00:58:29.228872 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-2c17eab1-d888-4b61-8afe-517831ce5374 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.401517 Sep 04 00:58:29.228872 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 19/39] 10.222.0.42 () {60 vars in 1190 bytes} [Sun Sep 4 00:58:28 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 403 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 04 00:58:29.234345 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-5b311939-d5e5-4290-9933-7ca786b01894 admin admin] Calling method '>' {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:58:29.238696 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG dbcounter [-] [106007] Writing DB stats nova_api:SELECT=47,nova_api:SAVEPOINT=5,nova_api:INSERT=10,nova_api:RELEASE=5 {{(pid=106007) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:58:29.258819 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-5b311939-d5e5-4290-9933-7ca786b01894 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.027997 Sep 04 00:58:29.259367 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 21/40] 10.222.0.42 () {60 vars in 1210 bytes} [Sun Sep 4 00:58:29 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 00:58:29.269026 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-c30686ca-0c32-4ccf-9e01-08bed16a2493 admin admin] Calling method '>' {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:58:29.291860 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-c30686ca-0c32-4ccf-9e01-08bed16a2493 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.026417 Sep 04 00:58:29.292389 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 20/41] 10.222.0.42 () {60 vars in 1210 bytes} [Sun Sep 4 00:58:29 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 27 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 00:58:29.301582 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-d01a16ae-ac51-4aea-8425-416d3f5321cf admin admin] Calling method '>' {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:58:29.307616 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.wsgi [None req-d01a16ae-ac51-4aea-8425-416d3f5321cf admin admin] HTTP exception thrown: Flavor 42 could not be found. Sep 04 00:58:29.310850 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-d01a16ae-ac51-4aea-8425-416d3f5321cf admin admin] Request method failure captured: Sep 04 00:58:29.310850 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Sep 04 00:58:29.310850 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: Accept: application/json Sep 04 00:58:29.310850 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: Accept-Encoding: gzip, deflate Sep 04 00:58:29.310850 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: Connection: keep-alive Sep 04 00:58:29.310850 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: Content-Length: 0 Sep 04 00:58:29.310850 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: Host: 10.222.0.42 Sep 04 00:58:29.310850 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: Openstack-Api-Version: compute 2.92 Sep 04 00:58:29.310850 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: Openstack-System-Scope: None Sep 04 00:58:29.310850 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: User-Agent: python-novaclient Sep 04 00:58:29.310850 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Auth-Token: gAAAAABjE_g0cMpRNGHbAkzwYIyweFTOYfYfEWZzPrgBnFPkCF6hMTJ3RD2vtzflBBuBH9vdb_rsBoTS3KXVWHLDz57YLNSdAWmxc1AJ1Dt7QZgpeA8AseSIBrpa2RfWyrkP7o2Fn34bL3b67pKtPXssWKq_nUwiEDXwX1GT-bWsOOt9GZPvOBE Sep 04 00:58:29.310850 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Domain-Id: None Sep 04 00:58:29.310850 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Domain-Name: None Sep 04 00:58:29.310850 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Identity-Status: Confirmed Sep 04 00:58:29.310850 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Is-Admin-Project: True Sep 04 00:58:29.310850 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Openstack-Nova-Api-Version: 2.92 Sep 04 00:58:29.310850 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Project-Domain-Id: default Sep 04 00:58:29.310850 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Project-Domain-Name: Default Sep 04 00:58:29.310850 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Project-Id: f25649add4b1426590abafbe586fc1fa Sep 04 00:58:29.310850 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Project-Name: admin Sep 04 00:58:29.310850 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Role: reader,admin,member Sep 04 00:58:29.310850 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Roles: reader,admin,member Sep 04 00:58:29.318919 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/f25649add4b1426590abafbe586fc1fa"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_f25649add4b1426590abafbe586fc1fa", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/f25649add4b1426590abafbe586fc1fa"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/f25649add4b1426590abafbe586fc1fa"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 00:58:29.318919 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Tenant: admin Sep 04 00:58:29.318919 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Tenant-Id: f25649add4b1426590abafbe586fc1fa Sep 04 00:58:29.318919 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Tenant-Name: admin Sep 04 00:58:29.318919 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-User: admin Sep 04 00:58:29.318919 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-User-Domain-Id: default Sep 04 00:58:29.318919 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-User-Domain-Name: Default Sep 04 00:58:29.318919 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-User-Id: 01782590dd294bd488a6d5c08a043ce1 Sep 04 00:58:29.318919 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-User-Name: admin Sep 04 00:58:29.318919 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: method: > Sep 04 00:58:29.318919 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: exception: Flavor 42 could not be found. Sep 04 00:58:29.318919 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: action_args: {'id': '42'} Sep 04 00:58:29.318919 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 00:58:29.318919 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 00:58:29.320398 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 04 00:58:29.320398 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 04 00:58:29.320398 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 04 00:58:29.320398 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 04 00:58:29.320398 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 04 00:58:29.320398 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 04 00:58:29.320398 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 04 00:58:29.320398 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 04 00:58:29.320398 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 04 00:58:29.320398 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 00:58:29.320398 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 04 00:58:29.320398 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 04 00:58:29.320398 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Sep 04 00:58:29.320398 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi Sep 04 00:58:29.320398 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 00:58:29.320398 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi Sep 04 00:58:29.320398 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 00:58:29.320398 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 00:58:29.320398 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 00:58:29.322599 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 00:58:29.322599 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 00:58:29.322599 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 00:58:29.322599 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 00:58:29.322599 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 04 00:58:29.322599 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 04 00:58:29.322599 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Sep 04 00:58:29.322599 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi Sep 04 00:58:29.322599 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 00:58:29.322599 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi Sep 04 00:58:29.322599 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 00:58:29.322599 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 00:58:29.322599 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 00:58:29.322599 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 00:58:29.322599 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 00:58:29.322599 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Sep 04 00:58:29.322599 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi Sep 04 00:58:29.322599 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-d01a16ae-ac51-4aea-8425-416d3f5321cf admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=106007) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 00:58:29.324427 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-d01a16ae-ac51-4aea-8425-416d3f5321cf admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.014186 Sep 04 00:58:29.324427 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 22/42] 10.222.0.42 () {60 vars in 1190 bytes} [Sun Sep 4 00:58:29 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 15 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) Sep 04 00:58:30.268438 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-c30686ca-0c32-4ccf-9e01-08bed16a2493 admin admin] Calling method '>' {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:58:30.269550 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-c30686ca-0c32-4ccf-9e01-08bed16a2493 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001447 Sep 04 00:58:30.270747 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 21/43] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Sep 4 00:58:30 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 00:58:30.795628 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-2684277d-104d-4b00-88ce-afbb16f8e2d0 admin admin] Calling method '>' {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:58:30.800966 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.wsgi [None req-2684277d-104d-4b00-88ce-afbb16f8e2d0 admin admin] HTTP exception thrown: Flavor 84 could not be found. Sep 04 00:58:30.802577 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-2684277d-104d-4b00-88ce-afbb16f8e2d0 admin admin] Request method failure captured: Sep 04 00:58:30.802577 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Sep 04 00:58:30.802577 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: Accept: application/json Sep 04 00:58:30.802577 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: Accept-Encoding: gzip, deflate Sep 04 00:58:30.802577 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: Connection: keep-alive Sep 04 00:58:30.802577 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: Content-Length: 0 Sep 04 00:58:30.802577 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: Host: 10.222.0.42 Sep 04 00:58:30.802577 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: Openstack-Api-Version: compute 2.92 Sep 04 00:58:30.802577 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: Openstack-System-Scope: None Sep 04 00:58:30.802577 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: User-Agent: python-novaclient Sep 04 00:58:30.802577 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Auth-Token: gAAAAABjE_g2B87dz3miSTwFVdWlgYwppl-wQE2cX3j8y2EaZ87lBGNCwNj9_HA1lWk90CRd6V8DqqXU50U6K14uvSwSPPHZSJ2dqtKke_dqbZSFdgskC5BFlamX8uSPspAz3o1PqE3AlerU5Ul2H2_l4UKjjiI17ulrbkng30E6Nby9WtBFvCc Sep 04 00:58:30.802577 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Domain-Id: None Sep 04 00:58:30.802577 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Domain-Name: None Sep 04 00:58:30.802577 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Identity-Status: Confirmed Sep 04 00:58:30.802577 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Is-Admin-Project: True Sep 04 00:58:30.802577 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Openstack-Nova-Api-Version: 2.92 Sep 04 00:58:30.802577 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Project-Domain-Id: default Sep 04 00:58:30.802577 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Project-Domain-Name: Default Sep 04 00:58:30.802577 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Project-Id: f25649add4b1426590abafbe586fc1fa Sep 04 00:58:30.802577 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Project-Name: admin Sep 04 00:58:30.802577 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Role: reader,admin,member Sep 04 00:58:30.802577 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Roles: reader,admin,member Sep 04 00:58:30.803975 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/f25649add4b1426590abafbe586fc1fa"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_f25649add4b1426590abafbe586fc1fa", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/f25649add4b1426590abafbe586fc1fa"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/f25649add4b1426590abafbe586fc1fa"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 00:58:30.803975 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Tenant: admin Sep 04 00:58:30.803975 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Tenant-Id: f25649add4b1426590abafbe586fc1fa Sep 04 00:58:30.803975 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Tenant-Name: admin Sep 04 00:58:30.803975 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-User: admin Sep 04 00:58:30.803975 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-User-Domain-Id: default Sep 04 00:58:30.803975 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-User-Domain-Name: Default Sep 04 00:58:30.803975 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-User-Id: 01782590dd294bd488a6d5c08a043ce1 Sep 04 00:58:30.803975 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-User-Name: admin Sep 04 00:58:30.803975 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: method: > Sep 04 00:58:30.803975 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: exception: Flavor 84 could not be found. Sep 04 00:58:30.803975 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: action_args: {'id': '84'} Sep 04 00:58:30.803975 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 00:58:30.803975 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 00:58:30.804939 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 04 00:58:30.804939 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 04 00:58:30.804939 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 04 00:58:30.804939 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 04 00:58:30.804939 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 04 00:58:30.804939 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 04 00:58:30.804939 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 04 00:58:30.804939 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 04 00:58:30.804939 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 04 00:58:30.804939 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 00:58:30.804939 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 04 00:58:30.804939 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 04 00:58:30.804939 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Sep 04 00:58:30.804939 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi Sep 04 00:58:30.804939 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 00:58:30.804939 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi Sep 04 00:58:30.804939 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 00:58:30.804939 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 00:58:30.804939 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 00:58:30.806404 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 00:58:30.806404 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 00:58:30.806404 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 00:58:30.806404 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 00:58:30.806404 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 04 00:58:30.806404 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 04 00:58:30.806404 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Sep 04 00:58:30.806404 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi Sep 04 00:58:30.806404 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 00:58:30.806404 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi Sep 04 00:58:30.806404 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 00:58:30.806404 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 00:58:30.806404 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 00:58:30.806404 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 00:58:30.806404 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 00:58:30.806404 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Sep 04 00:58:30.806404 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi Sep 04 00:58:30.806404 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-2684277d-104d-4b00-88ce-afbb16f8e2d0 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=106007) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 00:58:30.807615 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-2684277d-104d-4b00-88ce-afbb16f8e2d0 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.396316 Sep 04 00:58:30.807615 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 23/44] 10.222.0.42 () {60 vars in 1190 bytes} [Sun Sep 4 00:58:30 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 397 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 04 00:58:30.813838 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-ef334873-0e85-4b56-8379-d7f82b9c9b21 admin admin] Calling method '>' {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:58:30.829361 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-ef334873-0e85-4b56-8379-d7f82b9c9b21 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019142 Sep 04 00:58:30.829889 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 22/45] 10.222.0.42 () {60 vars in 1210 bytes} [Sun Sep 4 00:58:30 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 00:58:30.839920 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-1eaed42a-fd38-4a56-a748-f4540ad16ecc admin admin] Calling method '>' {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:58:30.854628 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-1eaed42a-fd38-4a56-a748-f4540ad16ecc admin admin] 10.222.0.42 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018156 Sep 04 00:58:30.855157 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 24/46] 10.222.0.42 () {60 vars in 1210 bytes} [Sun Sep 4 00:58:30 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 19 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 00:58:30.864671 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-15d5d5d9-4476-497e-ab0b-53ff56c15c7d admin admin] Calling method '>' {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:58:30.870388 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.wsgi [None req-15d5d5d9-4476-497e-ab0b-53ff56c15c7d admin admin] HTTP exception thrown: Flavor 84 could not be found. Sep 04 00:58:30.871941 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-15d5d5d9-4476-497e-ab0b-53ff56c15c7d admin admin] Request method failure captured: Sep 04 00:58:30.871941 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Sep 04 00:58:30.871941 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: Accept: application/json Sep 04 00:58:30.871941 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: Accept-Encoding: gzip, deflate Sep 04 00:58:30.871941 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: Connection: keep-alive Sep 04 00:58:30.871941 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: Content-Length: 0 Sep 04 00:58:30.871941 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: Host: 10.222.0.42 Sep 04 00:58:30.871941 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: Openstack-Api-Version: compute 2.92 Sep 04 00:58:30.871941 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: Openstack-System-Scope: None Sep 04 00:58:30.871941 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: User-Agent: python-novaclient Sep 04 00:58:30.871941 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Auth-Token: gAAAAABjE_g2B87dz3miSTwFVdWlgYwppl-wQE2cX3j8y2EaZ87lBGNCwNj9_HA1lWk90CRd6V8DqqXU50U6K14uvSwSPPHZSJ2dqtKke_dqbZSFdgskC5BFlamX8uSPspAz3o1PqE3AlerU5Ul2H2_l4UKjjiI17ulrbkng30E6Nby9WtBFvCc Sep 04 00:58:30.871941 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Domain-Id: None Sep 04 00:58:30.871941 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Domain-Name: None Sep 04 00:58:30.871941 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Identity-Status: Confirmed Sep 04 00:58:30.871941 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Is-Admin-Project: True Sep 04 00:58:30.871941 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Openstack-Nova-Api-Version: 2.92 Sep 04 00:58:30.871941 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Project-Domain-Id: default Sep 04 00:58:30.871941 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Project-Domain-Name: Default Sep 04 00:58:30.871941 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Project-Id: f25649add4b1426590abafbe586fc1fa Sep 04 00:58:30.871941 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Project-Name: admin Sep 04 00:58:30.871941 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Role: reader,admin,member Sep 04 00:58:30.871941 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Roles: reader,admin,member Sep 04 00:58:30.873535 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/f25649add4b1426590abafbe586fc1fa"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_f25649add4b1426590abafbe586fc1fa", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/f25649add4b1426590abafbe586fc1fa"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/f25649add4b1426590abafbe586fc1fa"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 00:58:30.873535 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Tenant: admin Sep 04 00:58:30.873535 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Tenant-Id: f25649add4b1426590abafbe586fc1fa Sep 04 00:58:30.873535 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Tenant-Name: admin Sep 04 00:58:30.873535 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-User: admin Sep 04 00:58:30.873535 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-User-Domain-Id: default Sep 04 00:58:30.873535 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-User-Domain-Name: Default Sep 04 00:58:30.873535 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-User-Id: 01782590dd294bd488a6d5c08a043ce1 Sep 04 00:58:30.873535 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-User-Name: admin Sep 04 00:58:30.873535 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: method: > Sep 04 00:58:30.873535 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: exception: Flavor 84 could not be found. Sep 04 00:58:30.873535 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: action_args: {'id': '84'} Sep 04 00:58:30.873535 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 00:58:30.873535 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 00:58:30.874393 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 04 00:58:30.874393 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 04 00:58:30.874393 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 04 00:58:30.874393 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 04 00:58:30.874393 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 04 00:58:30.874393 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 04 00:58:30.874393 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 04 00:58:30.874393 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 04 00:58:30.874393 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 04 00:58:30.874393 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 00:58:30.874393 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 04 00:58:30.874393 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 04 00:58:30.874393 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Sep 04 00:58:30.874393 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi Sep 04 00:58:30.874393 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 00:58:30.874393 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi Sep 04 00:58:30.874393 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 00:58:30.874393 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 00:58:30.874393 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 00:58:30.875586 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 00:58:30.875586 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 00:58:30.875586 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 00:58:30.875586 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 00:58:30.875586 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 04 00:58:30.875586 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 04 00:58:30.875586 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Sep 04 00:58:30.875586 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi Sep 04 00:58:30.875586 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 00:58:30.875586 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi Sep 04 00:58:30.875586 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 00:58:30.875586 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 00:58:30.875586 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 00:58:30.875586 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 00:58:30.875586 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 00:58:30.875586 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Sep 04 00:58:30.875586 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi Sep 04 00:58:30.875586 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-15d5d5d9-4476-497e-ab0b-53ff56c15c7d admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=106008) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 00:58:30.876762 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-15d5d5d9-4476-497e-ab0b-53ff56c15c7d admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.012490 Sep 04 00:58:30.876762 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 23/47] 10.222.0.42 () {60 vars in 1190 bytes} [Sun Sep 4 00:58:30 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 14 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 04 00:58:31.839765 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-1eaed42a-fd38-4a56-a748-f4540ad16ecc admin admin] Calling method '>' {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:58:31.840764 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-1eaed42a-fd38-4a56-a748-f4540ad16ecc admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001422 Sep 04 00:58:31.841191 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 25/48] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Sep 4 00:58:31 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 00:58:32.371503 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-0f9f5fbe-c07d-4c53-852b-14c611f03db5 admin admin] Calling method '>' {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:58:32.377216 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.wsgi [None req-0f9f5fbe-c07d-4c53-852b-14c611f03db5 admin admin] HTTP exception thrown: Flavor 451 could not be found. Sep 04 00:58:32.378706 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-0f9f5fbe-c07d-4c53-852b-14c611f03db5 admin admin] Request method failure captured: Sep 04 00:58:32.378706 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Sep 04 00:58:32.378706 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: Accept: application/json Sep 04 00:58:32.378706 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: Accept-Encoding: gzip, deflate Sep 04 00:58:32.378706 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: Connection: keep-alive Sep 04 00:58:32.378706 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: Content-Length: 0 Sep 04 00:58:32.378706 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: Host: 10.222.0.42 Sep 04 00:58:32.378706 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: Openstack-Api-Version: compute 2.92 Sep 04 00:58:32.378706 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: Openstack-System-Scope: None Sep 04 00:58:32.378706 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: User-Agent: python-novaclient Sep 04 00:58:32.378706 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Auth-Token: gAAAAABjE_g3tMC3e5SW2lG57JxceeS0zbSBSup2Fw6CQdaN5DXg2_XO66D6QIsM_XmIK5QiV-w05jnNwPW9FCXAD9UJjDVm4I6sE4dQEMH4-TK6SW2AU5nsft-oTNzCUvFRRblQbcX7kGGPdNyX1BV6T7mB7FNZxLKjgiY-7HosvAInAvYS8hU Sep 04 00:58:32.378706 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Domain-Id: None Sep 04 00:58:32.378706 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Domain-Name: None Sep 04 00:58:32.378706 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Identity-Status: Confirmed Sep 04 00:58:32.378706 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Is-Admin-Project: True Sep 04 00:58:32.378706 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Openstack-Nova-Api-Version: 2.92 Sep 04 00:58:32.378706 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Project-Domain-Id: default Sep 04 00:58:32.378706 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Project-Domain-Name: Default Sep 04 00:58:32.378706 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Project-Id: f25649add4b1426590abafbe586fc1fa Sep 04 00:58:32.378706 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Project-Name: admin Sep 04 00:58:32.378706 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Role: reader,admin,member Sep 04 00:58:32.378706 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Roles: reader,admin,member Sep 04 00:58:32.380940 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/f25649add4b1426590abafbe586fc1fa"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_f25649add4b1426590abafbe586fc1fa", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/f25649add4b1426590abafbe586fc1fa"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/f25649add4b1426590abafbe586fc1fa"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 00:58:32.380940 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Tenant: admin Sep 04 00:58:32.380940 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Tenant-Id: f25649add4b1426590abafbe586fc1fa Sep 04 00:58:32.380940 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-Tenant-Name: admin Sep 04 00:58:32.380940 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-User: admin Sep 04 00:58:32.380940 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-User-Domain-Id: default Sep 04 00:58:32.380940 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-User-Domain-Name: Default Sep 04 00:58:32.380940 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-User-Id: 01782590dd294bd488a6d5c08a043ce1 Sep 04 00:58:32.380940 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: X-User-Name: admin Sep 04 00:58:32.380940 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: method: > Sep 04 00:58:32.380940 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: exception: Flavor 451 could not be found. Sep 04 00:58:32.380940 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: action_args: {'id': '451'} Sep 04 00:58:32.380940 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 00:58:32.380940 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 00:58:32.381858 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 04 00:58:32.381858 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 04 00:58:32.381858 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 04 00:58:32.381858 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 04 00:58:32.381858 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 04 00:58:32.381858 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 04 00:58:32.381858 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 04 00:58:32.381858 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 04 00:58:32.381858 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 04 00:58:32.381858 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 00:58:32.381858 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 04 00:58:32.381858 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 04 00:58:32.381858 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Sep 04 00:58:32.381858 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi Sep 04 00:58:32.381858 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 00:58:32.381858 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi Sep 04 00:58:32.381858 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 00:58:32.381858 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 00:58:32.381858 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 00:58:32.382969 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 00:58:32.382969 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 00:58:32.382969 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 00:58:32.382969 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 00:58:32.382969 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 04 00:58:32.382969 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 04 00:58:32.382969 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Sep 04 00:58:32.382969 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi Sep 04 00:58:32.382969 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 00:58:32.382969 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi Sep 04 00:58:32.382969 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 00:58:32.382969 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 00:58:32.382969 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 00:58:32.382969 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 00:58:32.382969 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 00:58:32.382969 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Sep 04 00:58:32.382969 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: ERROR nova.api.openstack.wsgi Sep 04 00:58:32.382969 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-0f9f5fbe-c07d-4c53-852b-14c611f03db5 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=106008) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 00:58:32.383992 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-0f9f5fbe-c07d-4c53-852b-14c611f03db5 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.395848 Sep 04 00:58:32.383992 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 24/49] 10.222.0.42 () {60 vars in 1193 bytes} [Sun Sep 4 00:58:31 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 397 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 04 00:58:32.390640 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-2cd6e93f-069a-4ff4-b367-24bb09291652 admin admin] Calling method '>' {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:58:32.406115 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-2cd6e93f-069a-4ff4-b367-24bb09291652 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019280 Sep 04 00:58:32.406641 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 26/50] 10.222.0.42 () {60 vars in 1210 bytes} [Sun Sep 4 00:58:32 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 00:58:32.415905 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-52380aea-b107-4ec4-b615-95a11d817aa4 admin admin] Calling method '>' {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:58:32.430626 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-52380aea-b107-4ec4-b615-95a11d817aa4 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018186 Sep 04 00:58:32.431185 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 25/51] 10.222.0.42 () {60 vars in 1210 bytes} [Sun Sep 4 00:58:32 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 19 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 00:58:32.440483 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-fa5e0edd-e6c2-4946-9624-1e0ad0290fa5 admin admin] Calling method '>' {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:58:32.446277 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.wsgi [None req-fa5e0edd-e6c2-4946-9624-1e0ad0290fa5 admin admin] HTTP exception thrown: Flavor 451 could not be found. Sep 04 00:58:32.447744 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-fa5e0edd-e6c2-4946-9624-1e0ad0290fa5 admin admin] Request method failure captured: Sep 04 00:58:32.447744 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Sep 04 00:58:32.447744 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: Accept: application/json Sep 04 00:58:32.447744 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: Accept-Encoding: gzip, deflate Sep 04 00:58:32.447744 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: Connection: keep-alive Sep 04 00:58:32.447744 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: Content-Length: 0 Sep 04 00:58:32.447744 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: Host: 10.222.0.42 Sep 04 00:58:32.447744 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: Openstack-Api-Version: compute 2.92 Sep 04 00:58:32.447744 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: Openstack-System-Scope: None Sep 04 00:58:32.447744 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: User-Agent: python-novaclient Sep 04 00:58:32.447744 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Auth-Token: gAAAAABjE_g3tMC3e5SW2lG57JxceeS0zbSBSup2Fw6CQdaN5DXg2_XO66D6QIsM_XmIK5QiV-w05jnNwPW9FCXAD9UJjDVm4I6sE4dQEMH4-TK6SW2AU5nsft-oTNzCUvFRRblQbcX7kGGPdNyX1BV6T7mB7FNZxLKjgiY-7HosvAInAvYS8hU Sep 04 00:58:32.447744 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Domain-Id: None Sep 04 00:58:32.447744 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Domain-Name: None Sep 04 00:58:32.447744 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Identity-Status: Confirmed Sep 04 00:58:32.447744 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Is-Admin-Project: True Sep 04 00:58:32.447744 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Openstack-Nova-Api-Version: 2.92 Sep 04 00:58:32.447744 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Project-Domain-Id: default Sep 04 00:58:32.447744 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Project-Domain-Name: Default Sep 04 00:58:32.447744 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Project-Id: f25649add4b1426590abafbe586fc1fa Sep 04 00:58:32.447744 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Project-Name: admin Sep 04 00:58:32.447744 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Role: reader,admin,member Sep 04 00:58:32.447744 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Roles: reader,admin,member Sep 04 00:58:32.449314 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/f25649add4b1426590abafbe586fc1fa"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_f25649add4b1426590abafbe586fc1fa", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/f25649add4b1426590abafbe586fc1fa"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/f25649add4b1426590abafbe586fc1fa"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 00:58:32.449314 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Tenant: admin Sep 04 00:58:32.449314 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Tenant-Id: f25649add4b1426590abafbe586fc1fa Sep 04 00:58:32.449314 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-Tenant-Name: admin Sep 04 00:58:32.449314 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-User: admin Sep 04 00:58:32.449314 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-User-Domain-Id: default Sep 04 00:58:32.449314 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-User-Domain-Name: Default Sep 04 00:58:32.449314 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-User-Id: 01782590dd294bd488a6d5c08a043ce1 Sep 04 00:58:32.449314 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: X-User-Name: admin Sep 04 00:58:32.449314 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: method: > Sep 04 00:58:32.449314 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: exception: Flavor 451 could not be found. Sep 04 00:58:32.449314 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: action_args: {'id': '451'} Sep 04 00:58:32.449314 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 00:58:32.449314 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 00:58:32.450237 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 04 00:58:32.450237 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 04 00:58:32.450237 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 04 00:58:32.450237 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 04 00:58:32.450237 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 04 00:58:32.450237 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 04 00:58:32.450237 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 04 00:58:32.450237 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 04 00:58:32.450237 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 04 00:58:32.450237 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 00:58:32.450237 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 04 00:58:32.450237 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 04 00:58:32.450237 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Sep 04 00:58:32.450237 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi Sep 04 00:58:32.450237 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 00:58:32.450237 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi Sep 04 00:58:32.450237 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 00:58:32.450237 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 00:58:32.450237 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 00:58:32.451541 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 00:58:32.451541 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 00:58:32.451541 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 00:58:32.451541 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 00:58:32.451541 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 04 00:58:32.451541 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 04 00:58:32.451541 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Sep 04 00:58:32.451541 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi Sep 04 00:58:32.451541 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 00:58:32.451541 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi Sep 04 00:58:32.451541 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 00:58:32.451541 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 00:58:32.451541 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 00:58:32.451541 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 00:58:32.451541 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 00:58:32.451541 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Sep 04 00:58:32.451541 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: ERROR nova.api.openstack.wsgi Sep 04 00:58:32.451541 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-fa5e0edd-e6c2-4946-9624-1e0ad0290fa5 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=106007) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 00:58:32.452814 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-fa5e0edd-e6c2-4946-9624-1e0ad0290fa5 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.012098 Sep 04 00:58:32.452814 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 27/52] 10.222.0.42 () {60 vars in 1193 bytes} [Sun Sep 4 00:58:32 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 04 00:58:33.439061 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-52380aea-b107-4ec4-b615-95a11d817aa4 admin admin] Calling method '>' {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:58:33.440097 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-52380aea-b107-4ec4-b615-95a11d817aa4 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001458 Sep 04 00:58:33.440505 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 26/53] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Sep 4 00:58:33 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 00:58:33.961068 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-624288fe-242c-4aed-8e13-6523320328d4 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 00:58:34.017255 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-624288fe-242c-4aed-8e13-6523320328d4 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.92 time: 0.438490 Sep 04 00:58:34.017831 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 28/54] 10.222.0.42 () {64 vars in 1235 bytes} [Sun Sep 4 00:58:33 2022] POST /compute/v2.1/flavors => generated 404 bytes in 440 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 04 00:58:34.990820 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-52380aea-b107-4ec4-b615-95a11d817aa4 admin admin] Calling method '>' {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:58:34.992082 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-52380aea-b107-4ec4-b615-95a11d817aa4 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001760 Sep 04 00:58:34.992484 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 27/55] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Sep 4 00:58:34 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 00:58:35.512151 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-3939f2fc-20b2-4a91-ae74-06c3de0b0854 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 00:58:35.589605 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-3939f2fc-20b2-4a91-ae74-06c3de0b0854 admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.92 time: 0.458778 Sep 04 00:58:35.590156 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 29/56] 10.222.0.42 () {64 vars in 1235 bytes} [Sun Sep 4 00:58:35 2022] POST /compute/v2.1/flavors => generated 405 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 04 00:58:36.589049 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-52380aea-b107-4ec4-b615-95a11d817aa4 admin admin] Calling method '>' {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:58:36.590148 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-52380aea-b107-4ec4-b615-95a11d817aa4 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001466 Sep 04 00:58:36.590546 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 28/57] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Sep 4 00:58:36 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 00:58:37.116839 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-116a6b7d-f6de-411e-91cc-4b63a95b887b admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 00:58:37.199005 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-116a6b7d-f6de-411e-91cc-4b63a95b887b admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.92 time: 0.467415 Sep 04 00:58:37.199542 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 30/58] 10.222.0.42 () {64 vars in 1235 bytes} [Sun Sep 4 00:58:36 2022] POST /compute/v2.1/flavors => generated 407 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 04 00:58:38.179977 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-52380aea-b107-4ec4-b615-95a11d817aa4 admin admin] Calling method '>' {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:58:38.180982 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-52380aea-b107-4ec4-b615-95a11d817aa4 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001429 Sep 04 00:58:38.181407 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 29/59] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Sep 4 00:58:38 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 00:58:38.823340 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-62d2d381-da62-44fc-82e1-ce7f5f2e127b admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 00:58:38.903144 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-62d2d381-da62-44fc-82e1-ce7f5f2e127b admin admin] 10.222.0.42 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.92 time: 0.583404 Sep 04 00:58:38.903630 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 31/60] 10.222.0.42 () {64 vars in 1235 bytes} [Sun Sep 4 00:58:38 2022] POST /compute/v2.1/flavors => generated 409 bytes in 585 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 04 00:58:39.875106 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-52380aea-b107-4ec4-b615-95a11d817aa4 admin admin] Calling method '>' {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:58:39.876125 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-52380aea-b107-4ec4-b615-95a11d817aa4 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001427 Sep 04 00:58:39.876558 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 30/61] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Sep 4 00:58:39 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 00:58:40.395474 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-8a1224ba-159c-4b57-9e7c-9e0b4f593126 admin admin] Calling method '>' {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:58:40.415614 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-8a1224ba-159c-4b57-9e7c-9e0b4f593126 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.92 time: 0.398731 Sep 04 00:58:40.416135 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 32/62] 10.222.0.42 () {60 vars in 1202 bytes} [Sun Sep 4 00:58:40 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 00:59:34.091298 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-52380aea-b107-4ec4-b615-95a11d817aa4 admin admin] Calling method '>' {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:59:34.092483 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-52380aea-b107-4ec4-b615-95a11d817aa4 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001531 Sep 04 00:59:34.092891 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 31/63] 10.222.0.42 () {54 vars in 931 bytes} [Sun Sep 4 00:59:34 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 00:59:34.103459 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG dbcounter [-] [106007] Writing DB stats nova_api:SELECT=26,nova_api:INSERT=4 {{(pid=106007) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:34.480350 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-a44ccbc3-acfe-481b-a60c-3cfcd7992b0b admin admin] Calling method '>' {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:59:34.501700 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-a44ccbc3-acfe-481b-a60c-3cfcd7992b0b admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.402727 Sep 04 00:59:34.502236 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 33/64] 10.222.0.42 () {60 vars in 1292 bytes} [Sun Sep 4 00:59:34 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 00:59:34.523042 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG dbcounter [-] [106008] Writing DB stats nova_api:INSERT=2,nova_api:SELECT=22,nova_api:SAVEPOINT=1,nova_api:RELEASE=1 {{(pid=106008) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 00:59:34.526766 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-18ce07ad-40e1-4ebf-8da6-274b2cbbfbb9 admin admin] Calling method '>' {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:59:34.534891 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-18ce07ad-40e1-4ebf-8da6-274b2cbbfbb9 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.013225 Sep 04 00:59:34.535411 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 32/65] 10.222.0.42 () {60 vars in 1283 bytes} [Sun Sep 4 00:59:34 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 00:59:34.548135 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-a735e8a7-ec83-418f-8fff-c27cf841be6e admin admin] Calling method '>' {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:59:34.555735 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-a735e8a7-ec83-418f-8fff-c27cf841be6e admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011817 Sep 04 00:59:34.556236 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 34/66] 10.222.0.42 () {60 vars in 1286 bytes} [Sun Sep 4 00:59:34 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 00:59:34.568016 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-63c87222-86a4-418f-a89b-d2d546bed327 admin admin] Calling method '>' {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:59:34.575598 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-63c87222-86a4-418f-a89b-d2d546bed327 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011403 Sep 04 00:59:34.576101 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 33/67] 10.222.0.42 () {60 vars in 1286 bytes} [Sun Sep 4 00:59:34 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 00:59:34.587585 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-f5201743-676e-4c42-b9b5-55b9804fcf17 admin admin] Calling method '>' {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 00:59:34.595616 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-f5201743-676e-4c42-b9b5-55b9804fcf17 admin admin] 10.222.0.42 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011512 Sep 04 00:59:34.596112 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 35/68] 10.222.0.42 () {60 vars in 1283 bytes} [Sun Sep 4 00:59:34 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:03:01.006600 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-63c87222-86a4-418f-a89b-d2d546bed327 admin admin] Calling method '>' {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:03:01.007649 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-63c87222-86a4-418f-a89b-d2d546bed327 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001467 Sep 04 01:03:01.008066 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 34/69] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Sep 4 01:03:01 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:03:01.153584 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG dbcounter [-] [106007] Writing DB stats nova_api:SELECT=6 {{(pid=106007) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:03:01.297358 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-b57053f9-e453-4828-bd2d-21b0e3eb235b admin admin] Calling method '>' {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:03:01.300689 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG oslo_concurrency.lockutils [None req-b57053f9-e453-4828-bd2d-21b0e3eb235b admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=106007) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:03:01.301187 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG oslo_concurrency.lockutils [None req-b57053f9-e453-4828-bd2d-21b0e3eb235b admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=106007) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:03:01.301616 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG oslo_concurrency.lockutils [None req-b57053f9-e453-4828-bd2d-21b0e3eb235b admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=106007) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:03:01.302317 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG oslo_concurrency.lockutils [None req-b57053f9-e453-4828-bd2d-21b0e3eb235b admin admin] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=106007) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:03:01.302763 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG oslo_concurrency.lockutils [None req-b57053f9-e453-4828-bd2d-21b0e3eb235b admin admin] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=106007) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:03:01.303172 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG oslo_concurrency.lockutils [None req-b57053f9-e453-4828-bd2d-21b0e3eb235b admin admin] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=106007) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:03:01.336904 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-b57053f9-e453-4828-bd2d-21b0e3eb235b admin admin] 10.222.0.42 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.185797 Sep 04 01:03:01.337477 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 36/70] 10.222.0.42 () {60 vars in 1193 bytes} [Sun Sep 4 01:03:01 2022] GET /compute/v2.1/os-services => generated 791 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 04 01:03:02.159640 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-63c87222-86a4-418f-a89b-d2d546bed327 admin admin] Calling method '>' {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:03:02.160648 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-63c87222-86a4-418f-a89b-d2d546bed327 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001450 Sep 04 01:03:02.161045 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 35/71] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Sep 4 01:03:02 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:03:02.351541 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-9cf472bb-9f28-4e37-9595-bc792c19c030 admin admin] Calling method '>' {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:03:02.355628 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG oslo_concurrency.lockutils [None req-9cf472bb-9f28-4e37-9595-bc792c19c030 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=106007) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:03:02.356096 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG oslo_concurrency.lockutils [None req-9cf472bb-9f28-4e37-9595-bc792c19c030 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=106007) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:03:02.356516 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG oslo_concurrency.lockutils [None req-9cf472bb-9f28-4e37-9595-bc792c19c030 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=106007) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:03:02.357248 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG oslo_concurrency.lockutils [None req-9cf472bb-9f28-4e37-9595-bc792c19c030 admin admin] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=106007) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:03:02.357702 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG oslo_concurrency.lockutils [None req-9cf472bb-9f28-4e37-9595-bc792c19c030 admin admin] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=106007) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:03:02.358107 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG oslo_concurrency.lockutils [None req-9cf472bb-9f28-4e37-9595-bc792c19c030 admin admin] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=106007) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:03:02.378506 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-9cf472bb-9f28-4e37-9595-bc792c19c030 admin admin] 10.222.0.42 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.077886 Sep 04 01:03:02.378983 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 37/72] 10.222.0.42 () {60 vars in 1193 bytes} [Sun Sep 4 01:03:02 2022] GET /compute/v2.1/os-services => generated 791 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 04 01:03:13.582007 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-63c87222-86a4-418f-a89b-d2d546bed327 admin admin] Calling method '>' {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:03:13.583011 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-63c87222-86a4-418f-a89b-d2d546bed327 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001426 Sep 04 01:03:13.583404 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 36/73] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Sep 4 01:03:13 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:03:13.727393 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG dbcounter [-] [106007] Writing DB stats nova_cell0:SELECT=4 {{(pid=106007) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:03:13.728147 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG dbcounter [-] [106007] Writing DB stats nova_cell1:SELECT=4 {{(pid=106007) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:03:13.728756 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG dbcounter [-] [106007] Writing DB stats nova_api:SELECT=8 {{(pid=106007) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:03:13.777440 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-e7ca7069-7754-40a2-a47b-59bd27545db5 admin admin] Calling method '>' {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:03:13.780769 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG oslo_concurrency.lockutils [None req-e7ca7069-7754-40a2-a47b-59bd27545db5 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=106007) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:03:13.781282 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG oslo_concurrency.lockutils [None req-e7ca7069-7754-40a2-a47b-59bd27545db5 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=106007) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:03:13.781710 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG oslo_concurrency.lockutils [None req-e7ca7069-7754-40a2-a47b-59bd27545db5 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=106007) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:03:13.782419 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG oslo_concurrency.lockutils [None req-e7ca7069-7754-40a2-a47b-59bd27545db5 admin admin] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=106007) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:03:13.782858 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG oslo_concurrency.lockutils [None req-e7ca7069-7754-40a2-a47b-59bd27545db5 admin admin] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=106007) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:03:13.783291 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG oslo_concurrency.lockutils [None req-e7ca7069-7754-40a2-a47b-59bd27545db5 admin admin] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=106007) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:03:13.808718 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-e7ca7069-7754-40a2-a47b-59bd27545db5 admin admin] 10.222.0.42 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.92 time: 0.085746 Sep 04 01:03:13.809300 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 38/74] 10.222.0.42 () {60 vars in 1193 bytes} [Sun Sep 4 01:03:13 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:03:14.648157 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: DEBUG nova.api.openstack.wsgi [None req-63c87222-86a4-418f-a89b-d2d546bed327 admin admin] Calling method '>' {{(pid=106008) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:03:14.649203 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: INFO nova.api.openstack.requestlog [None req-63c87222-86a4-418f-a89b-d2d546bed327 admin admin] 10.222.0.42 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001425 Sep 04 01:03:14.659890 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: [pid: 106008|app: 0|req: 37/75] 10.222.0.42 () {56 vars in 1074 bytes} [Sun Sep 4 01:03:14 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:03:14.843541 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG nova.api.openstack.wsgi [None req-838cfecd-071d-44da-92b0-57d5307257b8 admin admin] Calling method '>' {{(pid=106007) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:03:14.846861 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG oslo_concurrency.lockutils [None req-838cfecd-071d-44da-92b0-57d5307257b8 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=106007) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:03:14.847331 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG oslo_concurrency.lockutils [None req-838cfecd-071d-44da-92b0-57d5307257b8 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=106007) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:03:14.847755 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG oslo_concurrency.lockutils [None req-838cfecd-071d-44da-92b0-57d5307257b8 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=106007) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:03:14.848480 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG oslo_concurrency.lockutils [None req-838cfecd-071d-44da-92b0-57d5307257b8 admin admin] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=106007) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:03:14.848919 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG oslo_concurrency.lockutils [None req-838cfecd-071d-44da-92b0-57d5307257b8 admin admin] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=106007) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:03:14.849356 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: DEBUG oslo_concurrency.lockutils [None req-838cfecd-071d-44da-92b0-57d5307257b8 admin admin] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=106007) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:03:14.871680 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: INFO nova.api.openstack.requestlog [None req-838cfecd-071d-44da-92b0-57d5307257b8 admin admin] 10.222.0.42 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.92 time: 0.080790 Sep 04 01:03:14.872227 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: [pid: 106007|app: 0|req: 39/76] 10.222.0.42 () {60 vars in 1193 bytes} [Sun Sep 4 01:03:14 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:04:10.437856 dv-cinder-master-3-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... Sep 04 01:04:10.438932 dv-cinder-master-3-iscsi devstack@n-api.service[106006]: Sun Sep 4 01:04:10 2022 - graceful shutdown triggered... Sep 04 01:04:10.439380 dv-cinder-master-3-iscsi devstack@n-api.service[106007]: Gracefully killing worker 1 (pid: 106007)... Sep 04 01:04:10.439945 dv-cinder-master-3-iscsi devstack@n-api.service[106008]: Gracefully killing worker 2 (pid: 106008)... Sep 04 01:04:11.438879 dv-cinder-master-3-iscsi devstack@n-api.service[106006]: worker 1 buried after 1 seconds Sep 04 01:04:11.439120 dv-cinder-master-3-iscsi devstack@n-api.service[106006]: worker 2 buried after 1 seconds Sep 04 01:04:11.439120 dv-cinder-master-3-iscsi devstack@n-api.service[106006]: goodbye to uWSGI. Sep 04 01:04:11.440462 dv-cinder-master-3-iscsi systemd[1]: devstack@n-api.service: Succeeded. Sep 04 01:04:11.441004 dv-cinder-master-3-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. Sep 04 01:04:11.443723 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Sep 04 01:04:11.453870 dv-cinder-master-3-iscsi devstack@n-api.service[121017]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Sep 04 01:04:11.455839 dv-cinder-master-3-iscsi devstack@n-api.service[121017]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Sep 4 01:04:11 2022] *** Sep 04 01:04:11.455839 dv-cinder-master-3-iscsi devstack@n-api.service[121017]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 04 01:04:11.455839 dv-cinder-master-3-iscsi devstack@n-api.service[121017]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 04 01:04:11.455839 dv-cinder-master-3-iscsi devstack@n-api.service[121017]: nodename: dv-cinder-master-3-iscsi Sep 04 01:04:11.455839 dv-cinder-master-3-iscsi devstack@n-api.service[121017]: machine: x86_64 Sep 04 01:04:11.455839 dv-cinder-master-3-iscsi devstack@n-api.service[121017]: clock source: unix Sep 04 01:04:11.455839 dv-cinder-master-3-iscsi devstack@n-api.service[121017]: pcre jit disabled Sep 04 01:04:11.455839 dv-cinder-master-3-iscsi devstack@n-api.service[121017]: detected number of CPU cores: 4 Sep 04 01:04:11.455839 dv-cinder-master-3-iscsi devstack@n-api.service[121017]: current working directory: / Sep 04 01:04:11.455839 dv-cinder-master-3-iscsi devstack@n-api.service[121017]: detected binary path: /usr/bin/uwsgi-core Sep 04 01:04:11.455839 dv-cinder-master-3-iscsi devstack@n-api.service[121017]: your processes number limit is 47680 Sep 04 01:04:11.456553 dv-cinder-master-3-iscsi devstack@n-api.service[121017]: your memory page size is 4096 bytes Sep 04 01:04:11.456553 dv-cinder-master-3-iscsi devstack@n-api.service[121017]: detected max file descriptor number: 2048 Sep 04 01:04:11.456553 dv-cinder-master-3-iscsi devstack@n-api.service[121017]: lock engine: pthread robust mutexes Sep 04 01:04:11.456553 dv-cinder-master-3-iscsi devstack@n-api.service[121017]: thunder lock: enabled Sep 04 01:04:11.456553 dv-cinder-master-3-iscsi devstack@n-api.service[121017]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Sep 04 01:04:11.456553 dv-cinder-master-3-iscsi devstack@n-api.service[121017]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 04 01:04:11.497100 dv-cinder-master-3-iscsi devstack@n-api.service[121017]: Python main interpreter initialized at 0x5575d46269c0 Sep 04 01:04:11.497100 dv-cinder-master-3-iscsi devstack@n-api.service[121017]: python threads support enabled Sep 04 01:04:11.497100 dv-cinder-master-3-iscsi devstack@n-api.service[121017]: your server socket listen backlog is limited to 100 connections Sep 04 01:04:11.497100 dv-cinder-master-3-iscsi devstack@n-api.service[121017]: your mercy for graceful operations on workers is 90 seconds Sep 04 01:04:11.497100 dv-cinder-master-3-iscsi devstack@n-api.service[121017]: mapped 403077 bytes (393 KB) for 2 cores Sep 04 01:04:11.497100 dv-cinder-master-3-iscsi devstack@n-api.service[121017]: *** Operational MODE: preforking *** Sep 04 01:04:11.497675 dv-cinder-master-3-iscsi devstack@n-api.service[121017]: *** uWSGI is running in multiple interpreter mode *** Sep 04 01:04:11.497675 dv-cinder-master-3-iscsi devstack@n-api.service[121017]: spawned uWSGI master process (pid: 121017) Sep 04 01:04:11.497456 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Sep 04 01:04:11.501456 dv-cinder-master-3-iscsi devstack@n-api.service[121017]: spawned uWSGI worker 1 (pid: 121036, cores: 1) Sep 04 01:04:11.505154 dv-cinder-master-3-iscsi devstack@n-api.service[121017]: spawned uWSGI worker 2 (pid: 121037, cores: 1) Sep 04 01:04:11.505154 dv-cinder-master-3-iscsi devstack@n-api.service[121017]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 04 01:04:12.163478 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 04 01:04:12.174137 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 04 01:04:13.346530 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 04 01:04:13.346530 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: from cryptography.utils import int_from_bytes Sep 04 01:04:13.347199 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 04 01:04:13.347199 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: from cryptography.utils import int_from_bytes Sep 04 01:04:13.349130 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 04 01:04:13.349130 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: from cryptography.utils import int_from_bytes Sep 04 01:04:13.349589 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 04 01:04:13.349589 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: from cryptography.utils import int_from_bytes Sep 04 01:04:15.831618 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 04 01:04:15.832227 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 04 01:04:15.832437 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 04 01:04:15.832789 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 04 01:04:15.833307 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 04 01:04:15.833937 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.834277 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.834607 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.834933 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.835268 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.835582 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.836385 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.836708 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.837043 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.837341 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.837663 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.838167 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.838497 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.838816 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.839163 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.839533 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.840004 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.840354 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.840663 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.841196 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.841954 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.842269 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.842583 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.842926 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.843237 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.843556 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.843888 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.844191 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.844518 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.844831 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.845275 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.845590 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.845905 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.846213 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.846635 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.846946 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.847257 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.847578 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.847901 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.848236 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.848558 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.848879 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.849237 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.849557 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.849862 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.850174 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.850482 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.850802 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.851116 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.852414 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.852671 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.853216 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.853531 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.853887 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.854185 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.854438 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.854689 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.855009 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.855328 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.855636 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.855943 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.856248 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.856555 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.856861 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.857193 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.857501 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.857808 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.858138 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.858444 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.858767 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.859076 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.859395 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.859803 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.42 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.860053 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.42 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.860362 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.860708 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.861025 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.861374 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.861687 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.862274 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.862581 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.862889 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.863195 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.863513 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.863848 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.864152 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.864401 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.864703 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.865006 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.865504 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.866065 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.866454 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.866765 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.867252 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.867576 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.867949 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.868461 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.868761 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.869095 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.869409 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.869714 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.870023 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.870328 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.870635 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.870959 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.871278 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.871585 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.871896 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.872200 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.872505 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.872833 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.873258 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.873520 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.873843 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 04 01:04:15.874417 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.874561 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 04 01:04:15.874692 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.874692 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.874955 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 04 01:04:15.875098 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.875236 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 04 01:04:15.875409 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.875571 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 04 01:04:15.875716 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.875872 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.876024 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.876207 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.876361 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.876501 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.876682 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.876856 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.877006 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.877233 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.877233 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.877524 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.877524 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.877821 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.877964 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.878113 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.878266 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.878409 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.878563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.878705 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.878877 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.879030 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.879181 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.879329 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.879816 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.879981 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.880133 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.880283 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.880283 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.880572 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.880729 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.880939 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.881235 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.881235 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.881527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.881684 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.881828 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.881828 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.882222 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.882387 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.883276 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.883456 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.883612 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.883612 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.883898 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.884050 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.884205 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.884401 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.884559 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.884712 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.884863 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.885006 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.885006 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.885385 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.885583 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.885729 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.885871 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.886021 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.886182 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.886325 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.886477 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.886477 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.886761 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.886761 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.887040 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.887200 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.887343 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.887486 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.887637 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.887786 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.887940 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.888082 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.888222 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.888383 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.888544 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.888694 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.888847 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.888990 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.889246 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.889396 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.889551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.889952 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.890104 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.890255 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.890416 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.890416 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.890701 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.890859 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.891057 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.891345 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.891513 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.891669 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.891820 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.891971 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.892252 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.892403 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.892403 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.892687 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.892832 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.893032 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.893203 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.893358 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.893510 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.893745 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.893900 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.893900 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.894187 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.894336 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.894495 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.894645 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.894796 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.894947 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.895090 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.895232 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.895397 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.895540 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.895681 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.895829 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.895982 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.896124 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.896274 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.896434 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.896577 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.896724 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.896867 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.897018 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.897251 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.897421 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.897576 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.897718 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.897859 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.898016 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.898203 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.898853 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.899009 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.899158 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.899301 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.899528 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.899677 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.900001 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.42 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.900168 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.900321 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.42 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.900470 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.900626 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.900776 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.900926 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.901165 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.901334 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.901572 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.901732 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.901882 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.902036 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.902180 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.902180 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.902586 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.902730 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.902880 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.903030 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.903186 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.903345 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.903498 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.903651 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.903795 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.903944 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.904086 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.904234 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.904398 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.904552 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.904705 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.904848 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.904997 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.905215 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.905366 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.905508 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.905651 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.905798 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.905948 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.906103 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.906258 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.906401 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.906543 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.906684 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.906826 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.906976 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.907133 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.907294 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.907446 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.907596 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.907738 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.907879 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.908035 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.908189 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.908331 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.908481 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.908632 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.908774 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.908928 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.909096 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.909246 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.909389 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.909541 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.909684 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.909840 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.910093 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.910260 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.910415 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.910560 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.910716 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.910875 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.911025 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.911168 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.911318 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.911476 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.911709 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.911865 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.912016 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.912173 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.912337 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.912586 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.912739 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.912883 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.913038 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.913225 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.913370 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.913511 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.913652 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.913942 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.914097 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.914256 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.914411 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.914599 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.914744 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.914886 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.915027 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.915181 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.915335 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.915487 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.915487 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.917029 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.917242 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.917401 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.917546 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.917709 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.917854 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.917999 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.918151 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.918151 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.918431 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.918573 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.918723 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.918877 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.919033 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.919033 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.919302 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.919451 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.919613 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.919763 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.919919 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.920080 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.920325 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.920483 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.920627 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.920627 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.920920 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.921063 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.921232 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.921385 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.921527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.921779 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 04 01:04:15.922017 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.922167 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.922310 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.922459 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.922609 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.922609 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.922888 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.923048 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.923223 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.923378 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.923705 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.923890 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.924149 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.924305 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.924463 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.924615 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.924767 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.924910 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.925058 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.925249 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.925404 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.925554 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.925711 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.925863 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.926016 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.926167 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.926318 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.926506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.926662 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.926805 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.926947 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.927088 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.927228 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.927378 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.927573 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.927731 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.927880 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.928033 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.928175 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.928326 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.928477 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.928680 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.928835 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.928985 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.929144 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.929310 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.929484 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.929638 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.929838 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.929990 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.930142 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.930312 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.930543 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.930692 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.930860 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.931016 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.931157 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.931307 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.931449 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.931598 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.931747 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.931888 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.932144 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.932303 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.932454 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.932605 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.932756 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.932913 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.932913 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.933272 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.933424 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.933567 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.933717 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.933880 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.934033 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.934194 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.934345 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.934494 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.934644 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.934644 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.934938 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.935088 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.935237 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.935387 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.935537 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.935690 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.935840 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.935984 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.936137 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.936286 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.936435 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.936587 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.936738 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.936882 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.937030 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.937209 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.937361 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.937513 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.937662 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.937813 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.937962 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.938123 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.938272 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.938424 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.938565 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.938706 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.938855 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.938997 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.939139 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.939290 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.939438 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.939589 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.939736 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.939885 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.940051 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.940202 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.940351 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.940509 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.940741 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.940898 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.941055 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.941283 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.941427 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.941569 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.941710 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.941852 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.942014 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.942249 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.942407 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.942550 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.942699 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.942851 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.943004 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.943156 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.943306 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.943449 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.943590 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.943740 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.943890 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.944043 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.944186 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.944328 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.944469 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.944621 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.944770 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.944920 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.945623 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.945770 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.945959 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.946123 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.946276 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.946428 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.946428 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.946715 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.946865 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.947021 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.947176 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.947321 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.947463 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.947604 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.947759 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.947939 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.948086 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.948228 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.948389 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.948540 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.948697 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.948848 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.948991 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.949287 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.949454 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.949606 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.949606 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.949606 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.950026 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.950177 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.950319 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.950480 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.950633 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.950881 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.951031 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.951182 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.951332 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.951558 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.951718 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.951862 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.952011 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.952396 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.952557 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.952903 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.953106 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.953267 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.953422 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.953583 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.953735 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.953878 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.954028 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.954179 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.954330 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.954481 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.954634 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.954785 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.954935 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.955096 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.955253 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.955402 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.955554 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.955706 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.955848 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.955997 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.956153 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.956303 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.956453 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.956596 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.956746 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.956906 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.957062 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.957062 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.957062 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.957506 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.957657 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.957657 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.957942 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.958084 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.958084 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.958357 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.958506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.958657 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.958810 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.958952 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.959101 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.959101 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.959385 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.959537 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.959678 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.959829 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.959972 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.960122 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.960263 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.960418 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.960568 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.961652 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.961815 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.961966 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.962192 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.962352 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.962611 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.962771 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.962922 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.963075 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.963228 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.963379 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.963522 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.963746 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.963746 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.964051 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.964208 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.964361 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.964504 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.964504 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.964771 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.964912 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.964912 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.965235 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.965389 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.965550 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.965719 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.965874 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.966017 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.966171 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.966314 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.966456 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.966605 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.966605 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.966885 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.967035 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.967187 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.967330 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.967330 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.967628 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.967783 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.967934 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.968077 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.968219 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.968376 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.968528 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.968680 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.968680 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.968966 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.969126 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.969294 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.969439 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.969599 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.969742 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.969883 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.970026 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.970176 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.970328 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.970471 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.970620 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.970762 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.970904 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.971055 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.971207 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.971207 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.971497 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.971739 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.971890 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.972043 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.972186 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.972336 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.972480 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.972649 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.972882 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.973040 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.973266 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.973411 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.973573 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.973737 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.973900 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.974044 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.974044 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.974319 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.974463 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.974613 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.974769 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.974912 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.975061 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.975204 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.975346 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.975506 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.975703 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.975703 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.975982 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.976132 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.976332 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.976563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.976780 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.976930 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.976930 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.977400 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.977553 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.977708 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.977872 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.978023 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.978172 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.978323 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.978473 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.978473 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.978473 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.978473 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.978473 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.979146 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.979288 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.979439 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.979589 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.979745 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.979897 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.980049 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.980555 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.980709 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.980854 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.980997 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.981188 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.981345 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.981504 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.981933 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.982257 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.982420 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.982578 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.982729 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.982901 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.983133 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.983283 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.983433 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.983586 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.983586 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.983961 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.984117 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.984268 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.984418 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.984560 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.984727 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.984727 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.985006 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.985174 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.985318 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.985460 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.985614 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.985766 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.985908 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.986060 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.986209 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.986360 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.986360 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.986655 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.986805 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.987209 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.987362 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.987362 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.987640 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.987784 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.987925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.988066 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.988217 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.988361 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.988511 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.988665 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.988807 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.988956 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.989156 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.989319 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.989319 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.989596 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.989739 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.989889 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.990031 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.990182 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.990332 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.990481 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.990634 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.990776 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.990917 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.991186 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.991337 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.991337 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.991606 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.991758 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.991955 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.992281 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.992479 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.992632 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.992831 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.993026 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.993301 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.993464 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.993617 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.993776 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.993776 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.993776 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.994195 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.994338 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.994338 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.994623 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.994776 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.994776 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.995066 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.995210 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.995352 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.995518 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.995676 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.995841 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.995999 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.996149 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.996149 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.996149 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.996149 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.996149 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.996870 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.997029 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.997250 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.997413 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.997572 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.997723 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.997871 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.998018 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.998166 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.998324 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.998483 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.998634 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.998783 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.998950 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.999114 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.999272 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.999429 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.999586 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.999790 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:15.999950 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.000114 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.000265 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.000421 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.000579 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.000579 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.000915 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.001119 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.001290 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.001441 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.001590 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.001739 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.001888 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.002050 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.002220 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.002450 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.002615 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.002767 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.002933 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.003093 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.003262 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.003490 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.003653 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.003815 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.003980 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.004138 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.004290 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.004446 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.004595 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.004752 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.004752 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.005051 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.005285 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.005285 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.005597 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.005761 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.005912 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.006062 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.006218 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.006218 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.006510 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.006510 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.006807 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.006957 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.007107 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.007274 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.007432 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.008154 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.008154 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.008522 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.008698 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.008849 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.009210 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.009383 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.009535 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.009535 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.009817 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.009987 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.010152 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.010312 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.010462 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.010618 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.010767 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.010925 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.011089 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.011241 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.011398 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.011548 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.011696 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.011844 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.012017 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.012176 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.012326 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.012577 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.012737 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.012895 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.013060 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.013268 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.013268 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.013649 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.013807 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.013972 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.014181 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.014355 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.014514 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.014671 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.014828 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.014988 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.015145 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.015295 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.015453 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.015604 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.015761 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.015920 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.016070 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.016226 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.016386 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.016543 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.016692 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.016692 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.016990 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.017163 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.017325 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.017475 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.017634 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.017785 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.017934 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.018090 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.018090 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.018396 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.018555 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.018713 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.018863 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.019018 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.019177 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.019341 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.019500 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.019709 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.019861 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.020010 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.020159 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.020325 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.020566 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.020721 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.020879 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.021036 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.021250 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.021413 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.021565 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.021715 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.021872 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.022021 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.022177 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.022336 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.022498 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.022750 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.022908 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.023058 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.023276 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.024205 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.024375 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.024540 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.024705 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.024705 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.024705 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.024705 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.025318 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.025480 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.025480 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.025771 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.025922 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.026072 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.026221 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.026369 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.026518 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.026679 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.026854 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.026854 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.027723 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.027895 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.028056 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.028056 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.028356 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.028356 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.028724 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.028875 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.029039 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.029272 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.029433 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.029584 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.029742 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.029901 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.030075 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.030237 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.030394 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.030544 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.030693 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.030851 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.031011 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.031170 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.031319 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.031476 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.031627 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.031776 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.031934 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.032103 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.032255 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.032405 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.032554 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.032731 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.032977 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.033151 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.033320 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.033472 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.033628 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.033793 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.033793 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.034108 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.034348 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.034518 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.034679 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.034830 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.034987 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.035145 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.035295 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.035445 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.035604 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.035754 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.035910 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.036071 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.036222 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.036381 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.036544 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.036544 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.036828 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.036977 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.037165 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.037323 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.037323 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.037620 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.037769 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.037926 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.038087 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.038247 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.038406 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.038564 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.038715 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.038980 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.039135 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.039299 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.039458 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.039610 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.039768 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.039768 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.040072 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.040239 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.040391 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.040547 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.040706 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.040864 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.041024 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.041217 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.041379 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.041530 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.041688 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.041838 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.042001 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.042162 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.042319 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.042469 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.042618 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.042768 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.042947 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.043186 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.043358 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.043517 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.043517 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.043819 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.044053 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.044206 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.044481 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.044648 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.044800 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.044963 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.045153 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.045322 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.045474 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.045474 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.045766 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.045766 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.046109 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.046109 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.046413 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.046563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.046712 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.046872 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.047030 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.047180 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.047335 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.047493 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.047650 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.047809 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.048006 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.048168 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.048317 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.048485 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.048485 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.048787 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.048936 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.049106 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.049264 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.049414 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.049580 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.049739 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.049897 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.050048 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.050196 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.050196 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.050504 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.050656 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.050813 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.050963 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.051113 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.051270 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.051430 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.051588 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.051738 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.051887 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.052036 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.052196 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.052365 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.052543 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.052710 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.052869 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.052869 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.053229 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.053469 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.053634 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.053786 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.053786 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.054090 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.054258 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.054452 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.054702 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.054859 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.055017 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.055167 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.055326 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.055326 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.055609 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.055767 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.055925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.056084 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.056084 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.056377 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.056527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.056699 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.056851 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.057011 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.057227 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.057379 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.057528 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.057528 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.057821 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.058076 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.058076 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.058391 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.058540 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.058689 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.058858 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.059024 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.059175 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.059452 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.059621 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.059775 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.059935 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.060086 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.060244 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.060423 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.060590 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.060741 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.060910 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.061062 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.061283 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.061443 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.061607 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.061765 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.061930 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.062081 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.062230 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.062389 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.062547 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.062711 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.062876 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.063037 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.063188 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.063349 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.063599 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.063769 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.063921 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.064070 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.064229 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.064398 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.064558 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.064744 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.064981 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.064981 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.065377 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.065533 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.065692 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.065853 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.065853 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.066147 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.066454 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.066799 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.067119 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.067533 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.067874 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.068202 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.068398 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.068553 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.068713 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.068866 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.069018 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.069248 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.069402 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.069571 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.070855 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.071025 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.071188 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.071342 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.071508 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.071687 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.071687 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.071990 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.072143 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.072293 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.072443 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.072611 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.072762 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.072912 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.073063 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.073240 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.073403 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.073403 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.073830 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.073996 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.074158 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.074317 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.074479 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.074648 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.074800 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.074969 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.074969 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.075381 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.075548 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.075699 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.075860 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.076018 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.076169 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.076320 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.076543 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.076697 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.076943 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.077145 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.077317 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.077468 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.077631 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.077791 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.077961 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.078127 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.078308 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.078575 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.078744 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.078744 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.079051 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.079201 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.079359 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.079519 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.079677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.079827 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.079976 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.080147 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.080298 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.080458 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.080609 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.080759 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.080759 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.081050 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.081238 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.081400 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.081559 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.081709 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.081859 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.082008 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.082176 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.082328 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.082486 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.082638 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.082802 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.082953 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.083112 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.083270 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.083429 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.083580 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.083737 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.083985 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.084158 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.084158 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.084467 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.084619 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.084768 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.084916 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.085105 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.085389 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.085558 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.086265 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.086265 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.086579 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.086738 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.086917 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.087153 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.087321 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.087481 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.087645 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.087805 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.087956 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.088106 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.088254 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.088416 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.088598 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.088847 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.088847 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.088847 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.089374 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.089525 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.089689 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.089842 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.089992 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.090150 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.090301 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.090464 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.090616 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.090766 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.090924 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.091085 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.091292 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.091460 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.091631 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.091783 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.091934 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.092092 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.092244 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.092403 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.092503 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.092579 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.092579 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.092904 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.093098 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.093257 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.093593 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.093761 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.093900 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.094124 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.094240 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.094447 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.094763 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.094855 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.095077 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.095403 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.095525 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.095694 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.095998 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.096092 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.096324 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.096611 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.096713 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.096947 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.097244 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.097352 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.097612 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.097895 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.097992 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.098263 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.098502 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.098605 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.098922 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.099096 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.099234 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.099548 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.099796 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.099914 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.100091 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=121037) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 04 01:04:16.100295 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.100747 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.101287 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.101705 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.102142 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.102581 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.103016 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.103512 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.103987 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.104427 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.104876 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.105362 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.105807 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.106246 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.106700 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.107139 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.107804 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.108241 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.108710 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.109183 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.109627 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.110067 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.110502 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.110948 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.111390 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.111792 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.112495 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.112923 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.113370 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.113882 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.114246 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.114665 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.115103 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.115531 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.116191 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.116658 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.117231 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.117863 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.118314 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.118780 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.119207 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.119649 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.120075 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.120503 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.120938 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.121399 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.121816 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.122235 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.122667 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.123093 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.123514 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.123994 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.124381 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.124826 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.125344 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.125844 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.126195 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.126623 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.127045 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.127494 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.127877 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.128298 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.128740 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.129330 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.129705 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.130140 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.130567 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.131033 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.131504 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.131875 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.132313 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.133670 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.133670 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.134120 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.134546 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.134988 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.135415 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.135858 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.136326 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.136757 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.137460 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.137916 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.138352 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.138835 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.139259 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.139685 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.140092 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.140514 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.140952 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.141398 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.141849 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.142275 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.142702 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.143127 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.143552 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.143979 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.144410 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.144837 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.145791 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.145791 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.146493 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.146936 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.147416 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.147792 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.148840 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.149313 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.149795 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.150172 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.150608 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.151037 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.151461 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.151927 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.152301 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.152749 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.153234 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.153666 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.154090 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.154642 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.155073 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.155507 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.155944 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.156372 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.156808 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.157759 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.157759 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.158114 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.158539 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.158993 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.159712 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.160160 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.160604 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.161222 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.161714 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.162061 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.162392 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.162728 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.163055 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.163239 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO dbcounter [None req-daf1b4b4-950e-48ff-9f92-da233a7b1931 None None] Registered counter for database nova_api Sep 04 01:04:16.163376 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.163696 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.164173 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.164437 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.164741 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.165003 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.165333 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.165655 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.165970 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.166285 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.166603 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.166914 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.167228 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.167541 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.167861 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.168178 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.168491 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.168809 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.169144 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.169462 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.169807 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.170150 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.170475 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.170810 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.171133 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.171466 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.171785 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.172117 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.172451 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.172783 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.173123 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.173555 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.173785 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.174105 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.174428 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.174749 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.175066 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.175387 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.175722 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.176062 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.176376 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.176713 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.177039 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.177393 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.177717 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.178035 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.178587 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.178729 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.179008 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.179453 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.182665 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.182665 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.182665 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.182665 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.182665 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.182665 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.182665 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.182665 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.184359 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.184359 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.184359 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.184359 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.184359 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.185429 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_db.sqlalchemy.engines [None req-daf1b4b4-950e-48ff-9f92-da233a7b1931 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=121037) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 04 01:04:16.185637 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.185637 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.185637 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.185637 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.185637 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.186423 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 04 01:04:16.186423 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=121036) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 04 01:04:16.191209 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writer thread running {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 04 01:04:16.249219 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.context [None req-daf1b4b4-950e-48ff-9f92-da233a7b1931 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),cf96709b-43a7-4964-94cf-103a5b1a1016(cell1) {{(pid=121037) load_cells /opt/stack/nova/nova/context.py:464}} Sep 04 01:04:16.249413 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO dbcounter [None req-fc79a844-889f-45e1-9879-63ed289efb16 None None] Registered counter for database nova_api Sep 04 01:04:16.253107 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-daf1b4b4-950e-48ff-9f92-da233a7b1931 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:04:16.253714 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-daf1b4b4-950e-48ff-9f92-da233a7b1931 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:04:16.255440 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-daf1b4b4-950e-48ff-9f92-da233a7b1931 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:04:16.256192 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-daf1b4b4-950e-48ff-9f92-da233a7b1931 None None] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:04:16.256689 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-daf1b4b4-950e-48ff-9f92-da233a7b1931 None None] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:04:16.259557 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-daf1b4b4-950e-48ff-9f92-da233a7b1931 None None] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:04:16.261535 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO dbcounter [None req-daf1b4b4-950e-48ff-9f92-da233a7b1931 None None] Registered counter for database nova_cell0 Sep 04 01:04:16.265583 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO dbcounter [None req-daf1b4b4-950e-48ff-9f92-da233a7b1931 None None] Registered counter for database nova_cell1 Sep 04 01:04:16.269809 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_db.sqlalchemy.engines [None req-fc79a844-889f-45e1-9879-63ed289efb16 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=121036) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 04 01:04:16.272972 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_db.sqlalchemy.engines [None req-daf1b4b4-950e-48ff-9f92-da233a7b1931 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=121037) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 04 01:04:16.275448 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_db.sqlalchemy.engines [None req-daf1b4b4-950e-48ff-9f92-da233a7b1931 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=121037) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 04 01:04:16.276619 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writer thread running {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 04 01:04:16.281123 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writer thread running {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 04 01:04:16.283904 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writer thread running {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 04 01:04:16.334091 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.context [None req-fc79a844-889f-45e1-9879-63ed289efb16 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),cf96709b-43a7-4964-94cf-103a5b1a1016(cell1) {{(pid=121036) load_cells /opt/stack/nova/nova/context.py:464}} Sep 04 01:04:16.338461 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fc79a844-889f-45e1-9879-63ed289efb16 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:04:16.338942 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fc79a844-889f-45e1-9879-63ed289efb16 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:04:16.340509 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fc79a844-889f-45e1-9879-63ed289efb16 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:04:16.341299 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fc79a844-889f-45e1-9879-63ed289efb16 None None] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:04:16.341760 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fc79a844-889f-45e1-9879-63ed289efb16 None None] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:04:16.343775 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fc79a844-889f-45e1-9879-63ed289efb16 None None] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:04:16.345776 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO dbcounter [None req-fc79a844-889f-45e1-9879-63ed289efb16 None None] Registered counter for database nova_cell0 Sep 04 01:04:16.349787 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO dbcounter [None req-fc79a844-889f-45e1-9879-63ed289efb16 None None] Registered counter for database nova_cell1 Sep 04 01:04:16.357196 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_db.sqlalchemy.engines [None req-fc79a844-889f-45e1-9879-63ed289efb16 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=121036) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 04 01:04:16.359360 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_db.sqlalchemy.engines [None req-fc79a844-889f-45e1-9879-63ed289efb16 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=121036) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 04 01:04:16.365311 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writer thread running {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 04 01:04:16.366580 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writer thread running {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 04 01:04:16.434468 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO dbcounter [None req-daf1b4b4-950e-48ff-9f92-da233a7b1931 None None] Registered counter for database nova_cell0 Sep 04 01:04:16.441034 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_db.sqlalchemy.engines [None req-daf1b4b4-950e-48ff-9f92-da233a7b1931 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=121037) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 04 01:04:16.446425 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writer thread running {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 04 01:04:16.523353 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO dbcounter [None req-fc79a844-889f-45e1-9879-63ed289efb16 None None] Registered counter for database nova_cell0 Sep 04 01:04:16.531998 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_db.sqlalchemy.engines [None req-fc79a844-889f-45e1-9879-63ed289efb16 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=121036) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 04 01:04:16.536518 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writer thread running {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 04 01:04:16.769175 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: WARNING keystonemiddleware.auth_token [None req-daf1b4b4-950e-48ff-9f92-da233a7b1931 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 04 01:04:16.776597 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: WARNING keystonemiddleware.auth_token [None req-daf1b4b4-950e-48ff-9f92-da233a7b1931 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 04 01:04:16.863366 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: WARNING keystonemiddleware.auth_token [None req-fc79a844-889f-45e1-9879-63ed289efb16 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 04 01:04:16.871825 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: WARNING keystonemiddleware.auth_token [None req-fc79a844-889f-45e1-9879-63ed289efb16 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 04 01:04:17.130083 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: WARNING keystonemiddleware.auth_token [None req-daf1b4b4-950e-48ff-9f92-da233a7b1931 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 04 01:04:17.136658 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: WARNING keystonemiddleware.auth_token [None req-daf1b4b4-950e-48ff-9f92-da233a7b1931 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 04 01:04:17.142115 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5575d46269c0 pid: 121037 (default app) Sep 04 01:04:17.225192 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: WARNING keystonemiddleware.auth_token [None req-fc79a844-889f-45e1-9879-63ed289efb16 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 04 01:04:17.231952 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: WARNING keystonemiddleware.auth_token [None req-fc79a844-889f-45e1-9879-63ed289efb16 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 04 01:04:17.237581 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5575d46269c0 pid: 121036 (default app) Sep 04 01:04:53.824467 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_api:SELECT=3 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:04:53.826039 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell0:SELECT=3 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:04:53.826811 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=3 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:04:53.827561 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell0:SELECT=3 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:04:55.201566 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-94069f02-48de-4c14-8077-bbf96a1aba60 tempest-VolumesNegativeTest-124188684 tempest-VolumesNegativeTest-124188684-project] Action: 'create', calling method: .version_select at 0x7fc10d5d7310>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1514813496", "metadata": {"Type": "work"}}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:04:55.223864 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-94069f02-48de-4c14-8077-bbf96a1aba60 tempest-VolumesNegativeTest-124188684 tempest-VolumesNegativeTest-124188684-project] Request method failure captured: Sep 04 01:04:55.223864 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Sep 04 01:04:55.223864 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept: application/json Sep 04 01:04:55.223864 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept-Encoding: identity Sep 04 01:04:55.223864 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Connection: close Sep 04 01:04:55.223864 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Length: 122 Sep 04 01:04:55.223864 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Type: application/json Sep 04 01:04:55.223864 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Host: 10.222.0.42 Sep 04 01:04:55.223864 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Openstack-System-Scope: None Sep 04 01:04:55.223864 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: User-Agent: python-urllib3/1.26.12 Sep 04 01:04:55.223864 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Auth-Token: gAAAAABjE_m1pFsDejdLxq0AnNox2AqB4-q_vec5fZpgg3GhEOrECD_NsjuhFQqldLZecbxAH0XTQ7_hmb7POBXul-FLxl_vh0a4FkU_t_-0RQHc_euamCINL3mWPyiFwEeh47VLD6pVRK2-lJSRRS_nl_NzDBH3WRzIfhbP3XmUGsFHoeaXVWA Sep 04 01:04:55.223864 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Id: None Sep 04 01:04:55.223864 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Name: None Sep 04 01:04:55.223864 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Identity-Status: Confirmed Sep 04 01:04:55.223864 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Is-Admin-Project: True Sep 04 01:04:55.223864 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Id: default Sep 04 01:04:55.223864 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Name: Default Sep 04 01:04:55.223864 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Id: 959a493c8f8e49c990227d2229946a4f Sep 04 01:04:55.223864 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Name: tempest-VolumesNegativeTest-124188684 Sep 04 01:04:55.223864 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Role: reader,member Sep 04 01:04:55.223864 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Roles: reader,member Sep 04 01:04:55.226232 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/959a493c8f8e49c990227d2229946a4f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_959a493c8f8e49c990227d2229946a4f", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/959a493c8f8e49c990227d2229946a4f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/959a493c8f8e49c990227d2229946a4f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:04:55.226232 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant: tempest-VolumesNegativeTest-124188684 Sep 04 01:04:55.226232 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Id: 959a493c8f8e49c990227d2229946a4f Sep 04 01:04:55.226232 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Name: tempest-VolumesNegativeTest-124188684 Sep 04 01:04:55.226232 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User: tempest-VolumesNegativeTest-124188684-project Sep 04 01:04:55.226232 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Id: default Sep 04 01:04:55.226232 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Name: Default Sep 04 01:04:55.226232 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Id: 2fb029a637c74b47a227abf80b968af9 Sep 04 01:04:55.226232 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Name: tempest-VolumesNegativeTest-124188684-project Sep 04 01:04:55.226232 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Sep 04 01:04:55.226232 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1514813496", "metadata": {"Type": "work"}}} Sep 04 01:04:55.226232 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: method: Sep 04 01:04:55.226232 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Sep 04 01:04:55.227527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-1514813496', 'metadata': {'Type': 'work'}}}} Sep 04 01:04:55.227527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:04:55.227527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:04:55.227527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 04 01:04:55.227527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 04 01:04:55.227527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 04 01:04:55.227527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise error Sep 04 01:04:55.227527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Sep 04 01:04:55.227527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:04:55.227527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Sep 04 01:04:55.227527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Sep 04 01:04:55.227527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:04:55.227527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi On instance['volume']: Sep 04 01:04:55.227527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi '#$%' Sep 04 01:04:55.227527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:04:55.227527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:04:55.227527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:04:55.227527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:04:55.227527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:04:55.227527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:04:55.227527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:04:55.227527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:04:55.230839 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 04 01:04:55.230839 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 04 01:04:55.230839 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:04:55.230839 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:04:55.230839 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 04 01:04:55.230839 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 04 01:04:55.230839 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 04 01:04:55.230839 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 04 01:04:55.230839 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 04 01:04:55.230839 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 04 01:04:55.230839 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Sep 04 01:04:55.230839 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:04:55.230839 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:04:55.230839 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:04:55.230839 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:04:55.230839 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:04:55.230839 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:04:55.230839 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 04 01:04:55.230839 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 04 01:04:55.233860 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Sep 04 01:04:55.233860 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:04:55.233860 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-94069f02-48de-4c14-8077-bbf96a1aba60 tempest-VolumesNegativeTest-124188684 tempest-VolumesNegativeTest-124188684-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=121037) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:04:55.233860 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-94069f02-48de-4c14-8077-bbf96a1aba60 tempest-VolumesNegativeTest-124188684 tempest-VolumesNegativeTest-124188684-project] 10.222.0.42 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 1.404687 Sep 04 01:04:55.233860 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 1/1] 10.222.0.42 () {60 vars in 1156 bytes} [Sun Sep 4 01:04:53 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 1407 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Sep 04 01:04:55.236930 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_api:SELECT=3 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:04:55.238389 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=3 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:04:55.239202 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell0:SELECT=3 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:04:55.240374 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell0:SELECT=3 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:04:55.514405 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-e88317d2-5ba1-40f6-9737-c98bbce973d2 tempest-VolumesNegativeTest-124188684 tempest-VolumesNegativeTest-124188684-project] Action: 'create', calling method: .version_select at 0x7fc10d66ce50>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1669779635", "metadata": {"Type": "work"}}} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:04:55.521318 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-e88317d2-5ba1-40f6-9737-c98bbce973d2 tempest-VolumesNegativeTest-124188684 tempest-VolumesNegativeTest-124188684-project] Request method failure captured: Sep 04 01:04:55.521318 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Sep 04 01:04:55.521318 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept: application/json Sep 04 01:04:55.521318 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept-Encoding: identity Sep 04 01:04:55.521318 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Connection: close Sep 04 01:04:55.521318 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Length: 120 Sep 04 01:04:55.521318 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Type: application/json Sep 04 01:04:55.521318 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Host: 10.222.0.42 Sep 04 01:04:55.521318 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Openstack-System-Scope: None Sep 04 01:04:55.521318 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: User-Agent: python-urllib3/1.26.12 Sep 04 01:04:55.521318 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Auth-Token: gAAAAABjE_m1pFsDejdLxq0AnNox2AqB4-q_vec5fZpgg3GhEOrECD_NsjuhFQqldLZecbxAH0XTQ7_hmb7POBXul-FLxl_vh0a4FkU_t_-0RQHc_euamCINL3mWPyiFwEeh47VLD6pVRK2-lJSRRS_nl_NzDBH3WRzIfhbP3XmUGsFHoeaXVWA Sep 04 01:04:55.521318 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Id: None Sep 04 01:04:55.521318 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Name: None Sep 04 01:04:55.521318 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Identity-Status: Confirmed Sep 04 01:04:55.521318 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Is-Admin-Project: True Sep 04 01:04:55.521318 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Id: default Sep 04 01:04:55.521318 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Name: Default Sep 04 01:04:55.521318 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Id: 959a493c8f8e49c990227d2229946a4f Sep 04 01:04:55.521318 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Name: tempest-VolumesNegativeTest-124188684 Sep 04 01:04:55.521318 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Role: reader,member Sep 04 01:04:55.521318 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Roles: reader,member Sep 04 01:04:55.525332 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/959a493c8f8e49c990227d2229946a4f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_959a493c8f8e49c990227d2229946a4f", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/959a493c8f8e49c990227d2229946a4f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/959a493c8f8e49c990227d2229946a4f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:04:55.525332 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant: tempest-VolumesNegativeTest-124188684 Sep 04 01:04:55.525332 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Id: 959a493c8f8e49c990227d2229946a4f Sep 04 01:04:55.525332 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Name: tempest-VolumesNegativeTest-124188684 Sep 04 01:04:55.525332 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User: tempest-VolumesNegativeTest-124188684-project Sep 04 01:04:55.525332 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Id: default Sep 04 01:04:55.525332 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Name: Default Sep 04 01:04:55.525332 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Id: 2fb029a637c74b47a227abf80b968af9 Sep 04 01:04:55.525332 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Name: tempest-VolumesNegativeTest-124188684-project Sep 04 01:04:55.525332 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Sep 04 01:04:55.525332 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1669779635", "metadata": {"Type": "work"}}} Sep 04 01:04:55.525332 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: method: Sep 04 01:04:55.525332 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Sep 04 01:04:55.527146 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-1669779635', 'metadata': {'Type': 'work'}}}} Sep 04 01:04:55.527146 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:04:55.527146 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:04:55.527146 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 04 01:04:55.527146 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 04 01:04:55.527146 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 04 01:04:55.527146 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise error Sep 04 01:04:55.527146 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Sep 04 01:04:55.527146 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:04:55.527146 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Sep 04 01:04:55.527146 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Sep 04 01:04:55.527146 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:04:55.527146 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi On instance['volume']: Sep 04 01:04:55.527146 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi '0' Sep 04 01:04:55.527146 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:04:55.527146 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:04:55.527146 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:04:55.527146 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:04:55.527146 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:04:55.527146 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:04:55.527146 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:04:55.527146 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:04:55.528636 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 04 01:04:55.528636 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 04 01:04:55.528636 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:04:55.528636 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:04:55.528636 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 04 01:04:55.528636 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 04 01:04:55.528636 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 04 01:04:55.528636 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 04 01:04:55.528636 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 04 01:04:55.528636 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 04 01:04:55.528636 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Sep 04 01:04:55.528636 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:04:55.528636 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:04:55.528636 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:04:55.528636 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:04:55.528636 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:04:55.528636 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:04:55.528636 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 04 01:04:55.528636 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 04 01:04:55.529932 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Sep 04 01:04:55.529932 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:04:55.529932 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-e88317d2-5ba1-40f6-9737-c98bbce973d2 tempest-VolumesNegativeTest-124188684 tempest-VolumesNegativeTest-124188684-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=121036) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:04:55.529932 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-e88317d2-5ba1-40f6-9737-c98bbce973d2 tempest-VolumesNegativeTest-124188684 tempest-VolumesNegativeTest-124188684-project] 10.222.0.42 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.292590 Sep 04 01:04:55.529932 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 1/2] 10.222.0.42 () {60 vars in 1156 bytes} [Sun Sep 4 01:04:55 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 300 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 04 01:04:55.537745 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-2f29eb8d-6c7c-400b-95d6-f2fb1f8db4b8 tempest-VolumesNegativeTest-124188684 tempest-VolumesNegativeTest-124188684-project] Action: 'create', calling method: .version_select at 0x7fc10d5d7f70>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-543925042", "metadata": {"Type": "work"}}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:04:55.541448 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-2f29eb8d-6c7c-400b-95d6-f2fb1f8db4b8 tempest-VolumesNegativeTest-124188684 tempest-VolumesNegativeTest-124188684-project] Request method failure captured: Sep 04 01:04:55.541448 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Sep 04 01:04:55.541448 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept: application/json Sep 04 01:04:55.541448 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept-Encoding: identity Sep 04 01:04:55.541448 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Connection: close Sep 04 01:04:55.541448 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Length: 118 Sep 04 01:04:55.541448 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Type: application/json Sep 04 01:04:55.541448 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Host: 10.222.0.42 Sep 04 01:04:55.541448 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Openstack-System-Scope: None Sep 04 01:04:55.541448 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: User-Agent: python-urllib3/1.26.12 Sep 04 01:04:55.541448 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Auth-Token: gAAAAABjE_m1pFsDejdLxq0AnNox2AqB4-q_vec5fZpgg3GhEOrECD_NsjuhFQqldLZecbxAH0XTQ7_hmb7POBXul-FLxl_vh0a4FkU_t_-0RQHc_euamCINL3mWPyiFwEeh47VLD6pVRK2-lJSRRS_nl_NzDBH3WRzIfhbP3XmUGsFHoeaXVWA Sep 04 01:04:55.541448 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Id: None Sep 04 01:04:55.541448 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Name: None Sep 04 01:04:55.541448 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Identity-Status: Confirmed Sep 04 01:04:55.541448 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Is-Admin-Project: True Sep 04 01:04:55.541448 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Id: default Sep 04 01:04:55.541448 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Name: Default Sep 04 01:04:55.541448 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Id: 959a493c8f8e49c990227d2229946a4f Sep 04 01:04:55.541448 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Name: tempest-VolumesNegativeTest-124188684 Sep 04 01:04:55.541448 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Role: reader,member Sep 04 01:04:55.541448 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Roles: reader,member Sep 04 01:04:55.549285 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/959a493c8f8e49c990227d2229946a4f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_959a493c8f8e49c990227d2229946a4f", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/959a493c8f8e49c990227d2229946a4f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/959a493c8f8e49c990227d2229946a4f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:04:55.549285 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant: tempest-VolumesNegativeTest-124188684 Sep 04 01:04:55.549285 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Id: 959a493c8f8e49c990227d2229946a4f Sep 04 01:04:55.549285 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Name: tempest-VolumesNegativeTest-124188684 Sep 04 01:04:55.549285 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User: tempest-VolumesNegativeTest-124188684-project Sep 04 01:04:55.549285 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Id: default Sep 04 01:04:55.549285 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Name: Default Sep 04 01:04:55.549285 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Id: 2fb029a637c74b47a227abf80b968af9 Sep 04 01:04:55.549285 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Name: tempest-VolumesNegativeTest-124188684-project Sep 04 01:04:55.549285 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Sep 04 01:04:55.549285 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-543925042", "metadata": {"Type": "work"}}} Sep 04 01:04:55.549285 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: method: Sep 04 01:04:55.549285 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Sep 04 01:04:55.550667 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-543925042', 'metadata': {'Type': 'work'}}}} Sep 04 01:04:55.550667 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:04:55.550667 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:04:55.550667 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 04 01:04:55.550667 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 04 01:04:55.550667 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 04 01:04:55.550667 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise error Sep 04 01:04:55.550667 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Sep 04 01:04:55.550667 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:04:55.550667 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Sep 04 01:04:55.550667 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Sep 04 01:04:55.550667 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:04:55.550667 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi On instance['volume']: Sep 04 01:04:55.550667 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi '' Sep 04 01:04:55.550667 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:04:55.550667 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:04:55.550667 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:04:55.550667 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:04:55.550667 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:04:55.550667 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:04:55.550667 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:04:55.550667 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:04:55.551896 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 04 01:04:55.551896 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 04 01:04:55.551896 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:04:55.551896 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:04:55.551896 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 04 01:04:55.551896 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 04 01:04:55.551896 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 04 01:04:55.551896 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 04 01:04:55.551896 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 04 01:04:55.551896 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 04 01:04:55.551896 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Sep 04 01:04:55.551896 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:04:55.551896 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:04:55.551896 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:04:55.551896 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:04:55.551896 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:04:55.551896 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:04:55.551896 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 04 01:04:55.551896 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 04 01:04:55.554159 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Sep 04 01:04:55.554159 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:04:55.557016 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-2f29eb8d-6c7c-400b-95d6-f2fb1f8db4b8 tempest-VolumesNegativeTest-124188684 tempest-VolumesNegativeTest-124188684-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=121037) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:04:55.557967 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-2f29eb8d-6c7c-400b-95d6-f2fb1f8db4b8 tempest-VolumesNegativeTest-124188684 tempest-VolumesNegativeTest-124188684-project] 10.222.0.42 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.026353 Sep 04 01:04:55.559910 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 2/3] 10.222.0.42 () {60 vars in 1156 bytes} [Sun Sep 4 01:04:55 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 30 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 04 01:04:55.572179 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b4a31240-cbdc-4023-bee8-1e930e31cd95 tempest-VolumesNegativeTest-124188684 tempest-VolumesNegativeTest-124188684-project] Calling method '.version_select at 0x7fc10d635af0>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:04:56.449816 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.wsgi [None req-b4a31240-cbdc-4023-bee8-1e930e31cd95 tempest-VolumesNegativeTest-124188684 tempest-VolumesNegativeTest-124188684-project] HTTP exception thrown: Volume tempest-invalid-1546369792 could not be found. Sep 04 01:04:56.449816 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b4a31240-cbdc-4023-bee8-1e930e31cd95 tempest-VolumesNegativeTest-124188684 tempest-VolumesNegativeTest-124188684-project] Request method failure captured: Sep 04 01:04:56.449816 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-1546369792 HTTP/1.1 Sep 04 01:04:56.449816 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept: application/json Sep 04 01:04:56.449816 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept-Encoding: identity Sep 04 01:04:56.449816 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Connection: close Sep 04 01:04:56.449816 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Length: 0 Sep 04 01:04:56.449816 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Type: application/json Sep 04 01:04:56.449816 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Host: 10.222.0.42 Sep 04 01:04:56.449816 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Openstack-System-Scope: None Sep 04 01:04:56.449816 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: User-Agent: python-urllib3/1.26.12 Sep 04 01:04:56.449816 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Auth-Token: gAAAAABjE_m1pFsDejdLxq0AnNox2AqB4-q_vec5fZpgg3GhEOrECD_NsjuhFQqldLZecbxAH0XTQ7_hmb7POBXul-FLxl_vh0a4FkU_t_-0RQHc_euamCINL3mWPyiFwEeh47VLD6pVRK2-lJSRRS_nl_NzDBH3WRzIfhbP3XmUGsFHoeaXVWA Sep 04 01:04:56.449816 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Id: None Sep 04 01:04:56.449816 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Name: None Sep 04 01:04:56.449816 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Identity-Status: Confirmed Sep 04 01:04:56.449816 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Is-Admin-Project: True Sep 04 01:04:56.449816 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Id: default Sep 04 01:04:56.449816 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Name: Default Sep 04 01:04:56.449816 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Id: 959a493c8f8e49c990227d2229946a4f Sep 04 01:04:56.449816 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Name: tempest-VolumesNegativeTest-124188684 Sep 04 01:04:56.449816 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Role: reader,member Sep 04 01:04:56.449816 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Roles: reader,member Sep 04 01:04:56.454925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/959a493c8f8e49c990227d2229946a4f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_959a493c8f8e49c990227d2229946a4f", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/959a493c8f8e49c990227d2229946a4f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/959a493c8f8e49c990227d2229946a4f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:04:56.454925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant: tempest-VolumesNegativeTest-124188684 Sep 04 01:04:56.454925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Id: 959a493c8f8e49c990227d2229946a4f Sep 04 01:04:56.454925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Name: tempest-VolumesNegativeTest-124188684 Sep 04 01:04:56.454925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User: tempest-VolumesNegativeTest-124188684-project Sep 04 01:04:56.454925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Id: default Sep 04 01:04:56.454925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Name: Default Sep 04 01:04:56.454925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Id: 2fb029a637c74b47a227abf80b968af9 Sep 04 01:04:56.454925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Name: tempest-VolumesNegativeTest-124188684-project Sep 04 01:04:56.454925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: method: Sep 04 01:04:56.454925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: exception: Volume tempest-invalid-1546369792 could not be found. Sep 04 01:04:56.454925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: action_args: {'id': 'tempest-invalid-1546369792'} Sep 04 01:04:56.454925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:04:56.463645 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:04:56.463645 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 04 01:04:56.463645 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 04 01:04:56.463645 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 04 01:04:56.463645 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 04 01:04:56.463645 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 04 01:04:56.463645 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 04 01:04:56.463645 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 04 01:04:56.463645 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 04 01:04:56.463645 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 04 01:04:56.463645 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 04 01:04:56.463645 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 04 01:04:56.463645 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 04 01:04:56.463645 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 04 01:04:56.463645 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 04 01:04:56.463645 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-1546369792 could not be found. (HTTP 404) (Request-ID: req-e8748240-02e2-4608-bca4-90becc575e5c) Sep 04 01:04:56.463645 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:04:56.463645 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:04:56.463645 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:04:56.472770 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:04:56.472770 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Sep 04 01:04:56.472770 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Sep 04 01:04:56.472770 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 04 01:04:56.472770 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 04 01:04:56.472770 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 04 01:04:56.472770 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 04 01:04:56.472770 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 04 01:04:56.472770 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 04 01:04:56.472770 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 04 01:04:56.472770 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 04 01:04:56.472770 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 04 01:04:56.472770 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 04 01:04:56.472770 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 04 01:04:56.472770 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 04 01:04:56.472770 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 04 01:04:56.472770 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 04 01:04:56.472770 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 04 01:04:56.472770 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 04 01:04:56.475448 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 04 01:04:56.475448 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 04 01:04:56.475448 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 04 01:04:56.475448 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 04 01:04:56.475448 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-1546369792 could not be found. Sep 04 01:04:56.475448 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:04:56.475448 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:04:56.475448 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:04:56.475448 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:04:56.475448 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:04:56.475448 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:04:56.475448 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:04:56.475448 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:04:56.475448 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 04 01:04:56.475448 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 04 01:04:56.475448 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:04:56.475448 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:04:56.475448 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Sep 04 01:04:56.475448 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:04:56.475448 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-1546369792 could not be found. Sep 04 01:04:56.477614 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:04:56.477614 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:04:56.477614 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:04:56.477614 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:04:56.477614 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:04:56.477614 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:04:56.477614 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:04:56.477614 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:04:56.477614 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-1546369792 could not be found. Sep 04 01:04:56.477614 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:04:56.477614 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b4a31240-cbdc-4023-bee8-1e930e31cd95 tempest-VolumesNegativeTest-124188684 tempest-VolumesNegativeTest-124188684-project] Returning 404 to user: Volume tempest-invalid-1546369792 could not be found. {{(pid=121036) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:04:56.477614 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-b4a31240-cbdc-4023-bee8-1e930e31cd95 tempest-VolumesNegativeTest-124188684 tempest-VolumesNegativeTest-124188684-project] 10.222.0.42 "DELETE /compute/v2.1/os-volumes/tempest-invalid-1546369792" status: 404 len: 99 microversion: 2.1 time: 0.615061 Sep 04 01:04:56.477614 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 2/4] 10.222.0.42 () {58 vars in 1218 bytes} [Sun Sep 4 01:04:55 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-1546369792 => generated 99 bytes in 618 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 04 01:04:56.480258 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-8e32f485-e52e-4897-a31e-efab2fe253dd tempest-VolumesNegativeTest-124188684 tempest-VolumesNegativeTest-124188684-project] 10.222.0.42 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.020011 Sep 04 01:04:56.480258 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 3/5] 10.222.0.42 () {58 vars in 1140 bytes} [Sun Sep 4 01:04:56 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 24 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Sep 04 01:04:56.481232 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-c6000d9a-2f80-482b-a9d0-485384c900bf tempest-VolumesNegativeTest-124188684 tempest-VolumesNegativeTest-124188684-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.005487 Sep 04 01:04:56.481232 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 3/6] 10.222.0.42 () {58 vars in 1137 bytes} [Sun Sep 4 01:04:56 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 7 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Sep 04 01:04:56.483293 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-dd6d300c-bbc2-47c9-adbf-a65fb3d2e18e tempest-VolumesNegativeTest-124188684 tempest-VolumesNegativeTest-124188684-project] Calling method '.version_select at 0x7fc10d493310>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:04:56.537595 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.wsgi [None req-dd6d300c-bbc2-47c9-adbf-a65fb3d2e18e tempest-VolumesNegativeTest-124188684 tempest-VolumesNegativeTest-124188684-project] HTTP exception thrown: Volume cfad5548-b0ec-4e14-a939-4790cc288c91 could not be found. Sep 04 01:04:56.541529 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-dd6d300c-bbc2-47c9-adbf-a65fb3d2e18e tempest-VolumesNegativeTest-124188684 tempest-VolumesNegativeTest-124188684-project] Request method failure captured: Sep 04 01:04:56.541529 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: request: DELETE /compute/v2.1/os-volumes/cfad5548-b0ec-4e14-a939-4790cc288c91 HTTP/1.1 Sep 04 01:04:56.541529 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept: application/json Sep 04 01:04:56.541529 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept-Encoding: identity Sep 04 01:04:56.541529 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Connection: close Sep 04 01:04:56.541529 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Length: 0 Sep 04 01:04:56.541529 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Type: application/json Sep 04 01:04:56.541529 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Host: 10.222.0.42 Sep 04 01:04:56.541529 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Openstack-System-Scope: None Sep 04 01:04:56.541529 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: User-Agent: python-urllib3/1.26.12 Sep 04 01:04:56.541529 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Auth-Token: gAAAAABjE_m1pFsDejdLxq0AnNox2AqB4-q_vec5fZpgg3GhEOrECD_NsjuhFQqldLZecbxAH0XTQ7_hmb7POBXul-FLxl_vh0a4FkU_t_-0RQHc_euamCINL3mWPyiFwEeh47VLD6pVRK2-lJSRRS_nl_NzDBH3WRzIfhbP3XmUGsFHoeaXVWA Sep 04 01:04:56.541529 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Id: None Sep 04 01:04:56.541529 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Name: None Sep 04 01:04:56.541529 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Identity-Status: Confirmed Sep 04 01:04:56.541529 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Is-Admin-Project: True Sep 04 01:04:56.541529 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Id: default Sep 04 01:04:56.541529 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Name: Default Sep 04 01:04:56.541529 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Id: 959a493c8f8e49c990227d2229946a4f Sep 04 01:04:56.541529 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Name: tempest-VolumesNegativeTest-124188684 Sep 04 01:04:56.541529 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Role: reader,member Sep 04 01:04:56.541529 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Roles: reader,member Sep 04 01:04:56.542506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/959a493c8f8e49c990227d2229946a4f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_959a493c8f8e49c990227d2229946a4f", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/959a493c8f8e49c990227d2229946a4f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/959a493c8f8e49c990227d2229946a4f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:04:56.542506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant: tempest-VolumesNegativeTest-124188684 Sep 04 01:04:56.542506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Id: 959a493c8f8e49c990227d2229946a4f Sep 04 01:04:56.542506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Name: tempest-VolumesNegativeTest-124188684 Sep 04 01:04:56.542506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User: tempest-VolumesNegativeTest-124188684-project Sep 04 01:04:56.542506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Id: default Sep 04 01:04:56.542506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Name: Default Sep 04 01:04:56.542506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Id: 2fb029a637c74b47a227abf80b968af9 Sep 04 01:04:56.542506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Name: tempest-VolumesNegativeTest-124188684-project Sep 04 01:04:56.542506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: method: Sep 04 01:04:56.542506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: exception: Volume cfad5548-b0ec-4e14-a939-4790cc288c91 could not be found. Sep 04 01:04:56.542506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: action_args: {'id': 'cfad5548-b0ec-4e14-a939-4790cc288c91'} Sep 04 01:04:56.542506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:04:56.543094 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:04:56.543094 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 04 01:04:56.543094 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 04 01:04:56.543094 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 04 01:04:56.543094 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 04 01:04:56.543094 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 04 01:04:56.543094 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 04 01:04:56.543094 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 04 01:04:56.543094 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 04 01:04:56.543094 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 04 01:04:56.543094 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 04 01:04:56.543094 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 04 01:04:56.543094 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 04 01:04:56.543094 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 04 01:04:56.543094 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 04 01:04:56.543094 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume cfad5548-b0ec-4e14-a939-4790cc288c91 could not be found. (HTTP 404) (Request-ID: req-56fbfd75-a0e3-406c-8b75-54d1974787da) Sep 04 01:04:56.543094 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:04:56.543094 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:04:56.543094 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:04:56.543891 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:04:56.543891 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Sep 04 01:04:56.543891 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Sep 04 01:04:56.543891 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 04 01:04:56.543891 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 04 01:04:56.543891 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 04 01:04:56.543891 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 04 01:04:56.543891 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 04 01:04:56.543891 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 04 01:04:56.543891 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 04 01:04:56.543891 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 04 01:04:56.543891 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 04 01:04:56.543891 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 04 01:04:56.543891 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 04 01:04:56.543891 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 04 01:04:56.543891 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 04 01:04:56.543891 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 04 01:04:56.543891 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 04 01:04:56.543891 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 04 01:04:56.544763 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 04 01:04:56.544763 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 04 01:04:56.544763 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 04 01:04:56.544763 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 04 01:04:56.544763 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume cfad5548-b0ec-4e14-a939-4790cc288c91 could not be found. Sep 04 01:04:56.544763 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:04:56.544763 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:04:56.544763 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:04:56.544763 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:04:56.544763 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:04:56.544763 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:04:56.544763 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:04:56.544763 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:04:56.544763 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 04 01:04:56.544763 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 04 01:04:56.544763 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:04:56.544763 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:04:56.544763 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Sep 04 01:04:56.544763 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:04:56.545587 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume cfad5548-b0ec-4e14-a939-4790cc288c91 could not be found. Sep 04 01:04:56.545587 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:04:56.545587 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:04:56.545587 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:04:56.545587 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:04:56.545587 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:04:56.545587 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:04:56.545587 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:04:56.545587 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:04:56.545587 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume cfad5548-b0ec-4e14-a939-4790cc288c91 could not be found. Sep 04 01:04:56.545587 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:04:56.545587 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-dd6d300c-bbc2-47c9-adbf-a65fb3d2e18e tempest-VolumesNegativeTest-124188684 tempest-VolumesNegativeTest-124188684-project] Returning 404 to user: Volume cfad5548-b0ec-4e14-a939-4790cc288c91 could not be found. {{(pid=121037) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:04:56.545587 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-dd6d300c-bbc2-47c9-adbf-a65fb3d2e18e tempest-VolumesNegativeTest-124188684 tempest-VolumesNegativeTest-124188684-project] 10.222.0.42 "DELETE /compute/v2.1/os-volumes/cfad5548-b0ec-4e14-a939-4790cc288c91" status: 404 len: 109 microversion: 2.1 time: 0.296448 Sep 04 01:04:56.548644 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 4/7] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Sep 4 01:04:56 2022] DELETE /compute/v2.1/os-volumes/cfad5548-b0ec-4e14-a939-4790cc288c91 => generated 109 bytes in 301 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 04 01:04:56.554728 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-cce32f78-d7d9-4934-aca4-816183280fde tempest-VolumesNegativeTest-124188684 tempest-VolumesNegativeTest-124188684-project] Calling method '.version_select at 0x7fc10d41e940>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:04:56.603434 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.wsgi [None req-cce32f78-d7d9-4934-aca4-816183280fde tempest-VolumesNegativeTest-124188684 tempest-VolumesNegativeTest-124188684-project] HTTP exception thrown: Volume e9aff761-a42f-42e4-87b9-87c74fb960ad could not be found. Sep 04 01:04:56.605631 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-cce32f78-d7d9-4934-aca4-816183280fde tempest-VolumesNegativeTest-124188684 tempest-VolumesNegativeTest-124188684-project] Request method failure captured: Sep 04 01:04:56.605631 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: request: GET /compute/v2.1/os-volumes/e9aff761-a42f-42e4-87b9-87c74fb960ad HTTP/1.1 Sep 04 01:04:56.605631 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept: application/json Sep 04 01:04:56.605631 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept-Encoding: identity Sep 04 01:04:56.605631 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Connection: close Sep 04 01:04:56.605631 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Length: 0 Sep 04 01:04:56.605631 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Type: application/json Sep 04 01:04:56.605631 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Host: 10.222.0.42 Sep 04 01:04:56.605631 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Openstack-System-Scope: None Sep 04 01:04:56.605631 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: User-Agent: python-urllib3/1.26.12 Sep 04 01:04:56.605631 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Auth-Token: gAAAAABjE_m1pFsDejdLxq0AnNox2AqB4-q_vec5fZpgg3GhEOrECD_NsjuhFQqldLZecbxAH0XTQ7_hmb7POBXul-FLxl_vh0a4FkU_t_-0RQHc_euamCINL3mWPyiFwEeh47VLD6pVRK2-lJSRRS_nl_NzDBH3WRzIfhbP3XmUGsFHoeaXVWA Sep 04 01:04:56.605631 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Id: None Sep 04 01:04:56.605631 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Name: None Sep 04 01:04:56.605631 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Identity-Status: Confirmed Sep 04 01:04:56.605631 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Is-Admin-Project: True Sep 04 01:04:56.605631 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Id: default Sep 04 01:04:56.605631 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Name: Default Sep 04 01:04:56.605631 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Id: 959a493c8f8e49c990227d2229946a4f Sep 04 01:04:56.605631 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Name: tempest-VolumesNegativeTest-124188684 Sep 04 01:04:56.605631 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Role: reader,member Sep 04 01:04:56.605631 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Roles: reader,member Sep 04 01:04:56.607766 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/959a493c8f8e49c990227d2229946a4f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_959a493c8f8e49c990227d2229946a4f", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/959a493c8f8e49c990227d2229946a4f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/959a493c8f8e49c990227d2229946a4f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:04:56.607766 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant: tempest-VolumesNegativeTest-124188684 Sep 04 01:04:56.607766 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Id: 959a493c8f8e49c990227d2229946a4f Sep 04 01:04:56.607766 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Name: tempest-VolumesNegativeTest-124188684 Sep 04 01:04:56.607766 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User: tempest-VolumesNegativeTest-124188684-project Sep 04 01:04:56.607766 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Id: default Sep 04 01:04:56.607766 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Name: Default Sep 04 01:04:56.607766 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Id: 2fb029a637c74b47a227abf80b968af9 Sep 04 01:04:56.607766 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Name: tempest-VolumesNegativeTest-124188684-project Sep 04 01:04:56.607766 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: method: Sep 04 01:04:56.607766 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: exception: Volume e9aff761-a42f-42e4-87b9-87c74fb960ad could not be found. Sep 04 01:04:56.607766 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: action_args: {'id': 'e9aff761-a42f-42e4-87b9-87c74fb960ad'} Sep 04 01:04:56.607766 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:04:56.608502 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:04:56.608502 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 04 01:04:56.608502 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 04 01:04:56.608502 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 04 01:04:56.608502 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 04 01:04:56.608502 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 04 01:04:56.608502 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 04 01:04:56.608502 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 04 01:04:56.608502 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 04 01:04:56.608502 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 04 01:04:56.608502 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 04 01:04:56.608502 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 04 01:04:56.608502 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 04 01:04:56.608502 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 04 01:04:56.608502 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 04 01:04:56.608502 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume e9aff761-a42f-42e4-87b9-87c74fb960ad could not be found. (HTTP 404) (Request-ID: req-c9aa270c-eb1c-40c7-ac68-2968be3790c5) Sep 04 01:04:56.608502 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:04:56.608502 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:04:56.608502 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:04:56.609591 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:04:56.609591 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 04 01:04:56.609591 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 04 01:04:56.609591 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 04 01:04:56.609591 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 04 01:04:56.609591 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 04 01:04:56.609591 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 04 01:04:56.609591 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 04 01:04:56.609591 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 04 01:04:56.609591 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 04 01:04:56.609591 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 04 01:04:56.609591 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 04 01:04:56.609591 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 04 01:04:56.609591 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 04 01:04:56.609591 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 04 01:04:56.609591 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 04 01:04:56.609591 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 04 01:04:56.609591 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 04 01:04:56.609591 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 04 01:04:56.610613 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 04 01:04:56.610613 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 04 01:04:56.610613 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 04 01:04:56.610613 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 04 01:04:56.610613 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume e9aff761-a42f-42e4-87b9-87c74fb960ad could not be found. Sep 04 01:04:56.610613 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:04:56.610613 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:04:56.610613 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:04:56.610613 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:04:56.610613 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:04:56.610613 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:04:56.610613 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:04:56.610613 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:04:56.610613 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 04 01:04:56.610613 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 04 01:04:56.610613 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:04:56.610613 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:04:56.610613 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 04 01:04:56.610613 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:04:56.612351 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume e9aff761-a42f-42e4-87b9-87c74fb960ad could not be found. Sep 04 01:04:56.612351 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:04:56.612351 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:04:56.612351 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:04:56.612351 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:04:56.612351 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:04:56.612351 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:04:56.612351 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:04:56.612351 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:04:56.612351 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume e9aff761-a42f-42e4-87b9-87c74fb960ad could not be found. Sep 04 01:04:56.612351 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:04:56.616111 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-cce32f78-d7d9-4934-aca4-816183280fde tempest-VolumesNegativeTest-124188684 tempest-VolumesNegativeTest-124188684-project] Returning 404 to user: Volume e9aff761-a42f-42e4-87b9-87c74fb960ad could not be found. {{(pid=121036) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:04:56.616111 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-cce32f78-d7d9-4934-aca4-816183280fde tempest-VolumesNegativeTest-124188684 tempest-VolumesNegativeTest-124188684-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/e9aff761-a42f-42e4-87b9-87c74fb960ad" status: 404 len: 109 microversion: 2.1 time: 0.064515 Sep 04 01:04:56.616111 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 4/8] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Sep 4 01:04:56 2022] GET /compute/v2.1/os-volumes/e9aff761-a42f-42e4-87b9-87c74fb960ad => generated 109 bytes in 66 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 04 01:05:08.546880 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-a3b14dc4-98b1-4011-8321-4f9aff03910d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-701353213", "imageRef": "a11315c3-ce81-478b-91e4-5353f189dfa7", "flavorRef": "452", "networks": [{"uuid": "3b7528ba-1624-4915-b635-aeb0700ea8e7"}]}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:05:08.759129 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.network.neutron [None req-a3b14dc4-98b1-4011-8321-4f9aff03910d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] validate_networks() for [('3b7528ba-1624-4915-b635-aeb0700ea8e7', None, None, None, None, None)] {{(pid=121037) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 04 01:05:11.507127 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-a3b14dc4-98b1-4011-8321-4f9aff03910d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Flavor limits 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 04 01:05:11.507798 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-a3b14dc4-98b1-4011-8321-4f9aff03910d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Image limits 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 04 01:05:11.508022 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-a3b14dc4-98b1-4011-8321-4f9aff03910d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Flavor pref 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 04 01:05:11.508436 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-a3b14dc4-98b1-4011-8321-4f9aff03910d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Image pref 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 04 01:05:11.508847 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-a3b14dc4-98b1-4011-8321-4f9aff03910d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 04 01:05:11.540763 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a3b14dc4-98b1-4011-8321-4f9aff03910d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:11.541303 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a3b14dc4-98b1-4011-8321-4f9aff03910d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:11.541750 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a3b14dc4-98b1-4011-8321-4f9aff03910d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:11.542480 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a3b14dc4-98b1-4011-8321-4f9aff03910d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:11.542932 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a3b14dc4-98b1-4011-8321-4f9aff03910d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:11.543354 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a3b14dc4-98b1-4011-8321-4f9aff03910d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:11.640712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.quota [None req-a3b14dc4-98b1-4011-8321-4f9aff03910d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Getting quotas for project e07d8b14fef34e04a0410f12e5dbd2fe. Resources: {'ram', 'instances', 'cores'} {{(pid=121037) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 04 01:05:11.665958 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.quota [None req-a3b14dc4-98b1-4011-8321-4f9aff03910d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Getting quotas for user b6b95b6814cf4cd8b77d87e7810c70e5 and project e07d8b14fef34e04a0410f12e5dbd2fe. Resources: {'ram', 'instances', 'cores'} {{(pid=121037) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 04 01:05:11.728525 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-a3b14dc4-98b1-4011-8321-4f9aff03910d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Going to run 1 instances... {{(pid=121037) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 04 01:05:11.728525 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-a3b14dc4-98b1-4011-8321-4f9aff03910d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a11315c3-ce81-478b-91e4-5353f189dfa7',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121037) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 04 01:05:11.997923 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-a3b14dc4-98b1-4011-8321-4f9aff03910d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 3.717474 Sep 04 01:05:11.999235 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 5/9] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 4 01:05:08 2022] POST /compute/v2.1/servers => generated 380 bytes in 3720 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 04 01:05:12.015773 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-cc5aac89-81b3-403e-8d1a-757a63c067ab tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:12.017222 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-cc5aac89-81b3-403e-8d1a-757a63c067ab tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:12.687648 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-cc5aac89-81b3-403e-8d1a-757a63c067ab tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:12.735423 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-cc5aac89-81b3-403e-8d1a-757a63c067ab tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1214 microversion: 2.1 time: 0.719756 Sep 04 01:05:12.736705 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 5/10] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:12 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1214 bytes in 726 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:13.783079 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-a46eea76-3631-43aa-bbb0-799290bdc13e tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:13.783079 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-a46eea76-3631-43aa-bbb0-799290bdc13e tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:13.951013 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-82ea5e72-7da8-4071-bfea-1d4b9a62a7ce tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Action: 'create', calling method: .version_select at 0x7fc10cad1280>, body: {"keypair": {"name": "tempest-keypair-677601389", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBGObuIeNigOuZu73ygEZlZMKXhMhdOWSLbl7yXw04ZeAQWXA/NHtx4OiqcWfNdDkB4MuBU0sclBx8DPly5a9x9Z5ms2zNESM5a0mq5z2Eo77bO48LZCCrR5xpWAurTEhBQ=="}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:05:14.018024 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-a46eea76-3631-43aa-bbb0-799290bdc13e tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:14.031890 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-a46eea76-3631-43aa-bbb0-799290bdc13e tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1214 microversion: 2.1 time: 0.254457 Sep 04 01:05:14.037032 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 6/11] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:13 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1214 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:05:14.047239 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.quota [None req-82ea5e72-7da8-4071-bfea-1d4b9a62a7ce tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Getting quotas for project e7dca68418a14d7da9aa63ac25f202e8. Resources: {'key_pairs'} {{(pid=121037) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 04 01:05:14.065882 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.quota [None req-82ea5e72-7da8-4071-bfea-1d4b9a62a7ce tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Getting quotas for user 44993cd93360442bb0afb28778ee83bc and project e7dca68418a14d7da9aa63ac25f202e8. Resources: {'key_pairs'} {{(pid=121037) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 04 01:05:14.226779 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-82ea5e72-7da8-4071-bfea-1d4b9a62a7ce tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.892371 Sep 04 01:05:14.236511 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 6/12] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Sep 4 01:05:13 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 904 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:05:15.065402 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-41268493-d0c1-4268-8e96-1172b7d61a6a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:15.066850 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-41268493-d0c1-4268-8e96-1172b7d61a6a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:15.291750 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-41268493-d0c1-4268-8e96-1172b7d61a6a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:15.350288 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-41268493-d0c1-4268-8e96-1172b7d61a6a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1214 microversion: 2.1 time: 0.268296 Sep 04 01:05:15.350965 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 7/13] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:15 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1214 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:15.550754 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-3bd7af2b-11e1-447d-9304-c020e2c3d2e1 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Action: 'create', calling method: .version_select at 0x7fc10d37d550>, body: {"keypair": {"name": "tempest-keypair-870969389", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKOap3BvfFfbC7Ahi/1+xIpi+Ml2bgyhEO5jXK85goIXUzG+vzzvCUHe+PhnrjLJjaSk2ZPGqkYdDxPd5Zh63T4xIxWgGi+njBpJ0SblvI5zYRFko/U8MwTvLhNLuuNvrA=="}} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:05:15.688536 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-3bd7af2b-11e1-447d-9304-c020e2c3d2e1 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Getting quotas for project 55faed42b7f243df9df879d581797f56. Resources: {'key_pairs'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 04 01:05:15.713410 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-3bd7af2b-11e1-447d-9304-c020e2c3d2e1 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Getting quotas for user fbcd4c19e766446188e3b3f41a44e2e4 and project 55faed42b7f243df9df879d581797f56. Resources: {'key_pairs'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 04 01:05:15.909968 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-3bd7af2b-11e1-447d-9304-c020e2c3d2e1 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 1.021832 Sep 04 01:05:15.918750 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 7/14] 10.222.0.42 () {62 vars in 1200 bytes} [Sun Sep 4 01:05:14 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 1024 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Sep 04 01:05:16.309523 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-808964f8-0111-4b15-8d4f-96754fec4fdf tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Action: 'create', calling method: .version_select at 0x7fc10cadf700>, body: {"keypair": {"name": "tempest-keypair-1366067967", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEVUNCtjZIWfBToK0CWa+KNq2wJ9sQYfKZHch+OJNq8rlT1dsCwuXYiK0c09nlPVod+bu9yfPf/AIz/0YoIwztbOUfPYOYsZHiVu/TJhAZ3dGbf+5srZOuOvajf8QvBa+g=="}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:05:16.353471 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.quota [None req-808964f8-0111-4b15-8d4f-96754fec4fdf tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Getting quotas for project 287d111f08ac4b05bd5adec9c40d0620. Resources: {'key_pairs'} {{(pid=121037) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 04 01:05:16.362665 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.quota [None req-808964f8-0111-4b15-8d4f-96754fec4fdf tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Getting quotas for user f5e4723258c64ce5885154dded2d3d43 and project 287d111f08ac4b05bd5adec9c40d0620. Resources: {'key_pairs'} {{(pid=121037) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 04 01:05:16.375485 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-7ff4657d-f04d-4c2f-9b3d-88e6af5536ad tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:16.376633 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-7ff4657d-f04d-4c2f-9b3d-88e6af5536ad tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:16.390281 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7ff4657d-f04d-4c2f-9b3d-88e6af5536ad tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:16.397514 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7ff4657d-f04d-4c2f-9b3d-88e6af5536ad tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:16.398011 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7ff4657d-f04d-4c2f-9b3d-88e6af5536ad tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:16.398768 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7ff4657d-f04d-4c2f-9b3d-88e6af5536ad tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:16.399231 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7ff4657d-f04d-4c2f-9b3d-88e6af5536ad tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:16.401011 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7ff4657d-f04d-4c2f-9b3d-88e6af5536ad tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:16.643764 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-7ff4657d-f04d-4c2f-9b3d-88e6af5536ad tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:16.657335 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-7ff4657d-f04d-4c2f-9b3d-88e6af5536ad tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1210 microversion: 2.1 time: 0.286092 Sep 04 01:05:16.669930 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 8/15] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:16 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1210 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:16.889110 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-7030d3dc-5a6f-41f7-a615-6faf3adf3b1d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1174546165", "imageRef": "a11315c3-ce81-478b-91e4-5353f189dfa7", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:05:16.968775 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.network.neutron [None req-7030d3dc-5a6f-41f7-a615-6faf3adf3b1d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] validate_networks() for [('b16f22e1-6fb1-440b-98eb-845123e960e6', None, None, None, None, None)] {{(pid=121036) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 04 01:05:17.637739 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-7030d3dc-5a6f-41f7-a615-6faf3adf3b1d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Flavor limits 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 04 01:05:17.638358 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-7030d3dc-5a6f-41f7-a615-6faf3adf3b1d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Image limits 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 04 01:05:17.638823 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-7030d3dc-5a6f-41f7-a615-6faf3adf3b1d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Flavor pref 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 04 01:05:17.639379 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-7030d3dc-5a6f-41f7-a615-6faf3adf3b1d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Image pref 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 04 01:05:17.639960 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-7030d3dc-5a6f-41f7-a615-6faf3adf3b1d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 04 01:05:17.650244 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7030d3dc-5a6f-41f7-a615-6faf3adf3b1d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:17.651368 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7030d3dc-5a6f-41f7-a615-6faf3adf3b1d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:17.651982 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7030d3dc-5a6f-41f7-a615-6faf3adf3b1d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:17.652700 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7030d3dc-5a6f-41f7-a615-6faf3adf3b1d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:17.653168 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7030d3dc-5a6f-41f7-a615-6faf3adf3b1d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:17.653740 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7030d3dc-5a6f-41f7-a615-6faf3adf3b1d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:17.675978 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-7030d3dc-5a6f-41f7-a615-6faf3adf3b1d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Getting quotas for project db72b63281384bd883b04d05bcc35677. Resources: {'ram', 'instances', 'cores'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 04 01:05:17.680029 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-7030d3dc-5a6f-41f7-a615-6faf3adf3b1d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Getting quotas for user bd13df3a2d0e4182bbd86dab7035c371 and project db72b63281384bd883b04d05bcc35677. Resources: {'ram', 'instances', 'cores'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 04 01:05:17.691198 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-7030d3dc-5a6f-41f7-a615-6faf3adf3b1d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Going to run 1 instances... {{(pid=121036) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 04 01:05:17.694530 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-7030d3dc-5a6f-41f7-a615-6faf3adf3b1d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a11315c3-ce81-478b-91e4-5353f189dfa7',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121036) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 04 01:05:17.807148 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-808964f8-0111-4b15-8d4f-96754fec4fdf tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 2.438745 Sep 04 01:05:17.808680 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 8/16] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Sep 4 01:05:15 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 2457 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:05:17.815570 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-098425be-bd76-4796-95ae-b71e65d1e2a9 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:17.817980 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-098425be-bd76-4796-95ae-b71e65d1e2a9 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:17.829139 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-098425be-bd76-4796-95ae-b71e65d1e2a9 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:17.829902 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-098425be-bd76-4796-95ae-b71e65d1e2a9 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:17.830748 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-098425be-bd76-4796-95ae-b71e65d1e2a9 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:17.831659 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-098425be-bd76-4796-95ae-b71e65d1e2a9 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:17.832365 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-098425be-bd76-4796-95ae-b71e65d1e2a9 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:17.833242 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-098425be-bd76-4796-95ae-b71e65d1e2a9 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:17.873112 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-7030d3dc-5a6f-41f7-a615-6faf3adf3b1d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.117326 Sep 04 01:05:17.887536 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 9/17] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 4 01:05:16 2022] POST /compute/v2.1/servers => generated 383 bytes in 1133 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 04 01:05:17.914485 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-fd9a5e4b-2d79-4a1e-a85a-abbe3ee5242f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:17.915120 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-fd9a5e4b-2d79-4a1e-a85a-abbe3ee5242f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:18.091726 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-098425be-bd76-4796-95ae-b71e65d1e2a9 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:18.114943 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-098425be-bd76-4796-95ae-b71e65d1e2a9 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1266 microversion: 2.1 time: 0.304825 Sep 04 01:05:18.117746 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 9/18] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:17 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1266 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:18.135186 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-fd9a5e4b-2d79-4a1e-a85a-abbe3ee5242f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:18.159037 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-fd9a5e4b-2d79-4a1e-a85a-abbe3ee5242f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1222 microversion: 2.1 time: 0.265244 Sep 04 01:05:18.161536 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 10/19] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:17 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1222 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:18.538351 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-51029590-2d32-45af-9a12-3914a029d26e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-90198257", "imageRef": "a11315c3-ce81-478b-91e4-5353f189dfa7", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:05:18.667579 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.network.neutron [None req-51029590-2d32-45af-9a12-3914a029d26e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] validate_networks() for [('b16f22e1-6fb1-440b-98eb-845123e960e6', None, None, None, None, None)] {{(pid=121037) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 04 01:05:19.142026 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b565a161-10c1-408f-a70f-41244e9cc9d9 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:19.142770 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-b565a161-10c1-408f-a70f-41244e9cc9d9 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:19.150243 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b565a161-10c1-408f-a70f-41244e9cc9d9 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:19.161442 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b565a161-10c1-408f-a70f-41244e9cc9d9 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:19.166728 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b565a161-10c1-408f-a70f-41244e9cc9d9 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:19.169015 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b565a161-10c1-408f-a70f-41244e9cc9d9 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:19.173925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b565a161-10c1-408f-a70f-41244e9cc9d9 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:19.174700 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b565a161-10c1-408f-a70f-41244e9cc9d9 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:19.528288 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-b565a161-10c1-408f-a70f-41244e9cc9d9 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:19.538022 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-b565a161-10c1-408f-a70f-41244e9cc9d9 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1274 microversion: 2.1 time: 0.400458 Sep 04 01:05:19.553535 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 11/20] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:19 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1274 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:05:20.577786 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-a9b6c461-418c-4d68-ac61-fab573a1fe1f tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:20.578678 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-a9b6c461-418c-4d68-ac61-fab573a1fe1f tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:20.595014 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a9b6c461-418c-4d68-ac61-fab573a1fe1f tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:20.599469 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a9b6c461-418c-4d68-ac61-fab573a1fe1f tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:20.599952 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a9b6c461-418c-4d68-ac61-fab573a1fe1f tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:20.603799 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a9b6c461-418c-4d68-ac61-fab573a1fe1f tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:20.604306 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a9b6c461-418c-4d68-ac61-fab573a1fe1f tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:20.607993 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a9b6c461-418c-4d68-ac61-fab573a1fe1f tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:20.886347 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-a9b6c461-418c-4d68-ac61-fab573a1fe1f tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:20.898124 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-a9b6c461-418c-4d68-ac61-fab573a1fe1f tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1272 microversion: 2.1 time: 0.325847 Sep 04 01:05:20.900862 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 12/21] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:20 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1272 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:21.592183 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell0:SELECT=2 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:05:21.928769 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-c223a70a-7569-4483-b385-e06da71187e5 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:21.930302 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-c223a70a-7569-4483-b385-e06da71187e5 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:21.940727 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c223a70a-7569-4483-b385-e06da71187e5 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:21.944693 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c223a70a-7569-4483-b385-e06da71187e5 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:21.945342 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c223a70a-7569-4483-b385-e06da71187e5 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:21.948166 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c223a70a-7569-4483-b385-e06da71187e5 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:21.948935 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c223a70a-7569-4483-b385-e06da71187e5 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:21.952997 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c223a70a-7569-4483-b385-e06da71187e5 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:22.005697 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-51029590-2d32-45af-9a12-3914a029d26e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Flavor limits 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 04 01:05:22.007444 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-51029590-2d32-45af-9a12-3914a029d26e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Image limits 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 04 01:05:22.007936 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-51029590-2d32-45af-9a12-3914a029d26e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Flavor pref 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 04 01:05:22.008535 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-51029590-2d32-45af-9a12-3914a029d26e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Image pref 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 04 01:05:22.009142 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-51029590-2d32-45af-9a12-3914a029d26e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 04 01:05:22.055208 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-51029590-2d32-45af-9a12-3914a029d26e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:22.057952 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-51029590-2d32-45af-9a12-3914a029d26e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:22.058434 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-51029590-2d32-45af-9a12-3914a029d26e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:22.064145 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-51029590-2d32-45af-9a12-3914a029d26e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:22.064638 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-51029590-2d32-45af-9a12-3914a029d26e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:22.065375 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-51029590-2d32-45af-9a12-3914a029d26e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:22.121735 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.quota [None req-51029590-2d32-45af-9a12-3914a029d26e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Getting quotas for project db72b63281384bd883b04d05bcc35677. Resources: {'ram', 'instances', 'cores'} {{(pid=121037) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 04 01:05:22.130312 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.quota [None req-51029590-2d32-45af-9a12-3914a029d26e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Getting quotas for user bd13df3a2d0e4182bbd86dab7035c371 and project db72b63281384bd883b04d05bcc35677. Resources: {'ram', 'instances', 'cores'} {{(pid=121037) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 04 01:05:22.169751 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-51029590-2d32-45af-9a12-3914a029d26e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Going to run 1 instances... {{(pid=121037) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 04 01:05:22.176279 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-51029590-2d32-45af-9a12-3914a029d26e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a11315c3-ce81-478b-91e4-5353f189dfa7',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121037) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 04 01:05:22.271860 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-c223a70a-7569-4483-b385-e06da71187e5 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:22.277296 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-c223a70a-7569-4483-b385-e06da71187e5 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1272 microversion: 2.1 time: 0.354303 Sep 04 01:05:22.277905 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 13/22] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:21 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1272 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:22.395847 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-51029590-2d32-45af-9a12-3914a029d26e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 3.863328 Sep 04 01:05:22.400270 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 10/23] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 4 01:05:18 2022] POST /compute/v2.1/servers => generated 383 bytes in 3869 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 04 01:05:22.416549 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-aa4fcb7e-b91e-4676-a5b4-09b7b081e73c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:22.417189 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-aa4fcb7e-b91e-4676-a5b4-09b7b081e73c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:22.871575 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-aa4fcb7e-b91e-4676-a5b4-09b7b081e73c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:22.901855 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-aa4fcb7e-b91e-4676-a5b4-09b7b081e73c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1220 microversion: 2.1 time: 0.480767 Sep 04 01:05:22.906078 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 14/24] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:22 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1220 bytes in 490 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:23.307493 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-ce92238b-cd43-40d0-8ff0-72be7b131ff7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:23.307493 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-ce92238b-cd43-40d0-8ff0-72be7b131ff7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:23.318821 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ce92238b-cd43-40d0-8ff0-72be7b131ff7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:23.328408 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ce92238b-cd43-40d0-8ff0-72be7b131ff7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:23.329109 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ce92238b-cd43-40d0-8ff0-72be7b131ff7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:23.334812 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ce92238b-cd43-40d0-8ff0-72be7b131ff7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:23.337241 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ce92238b-cd43-40d0-8ff0-72be7b131ff7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:23.337241 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ce92238b-cd43-40d0-8ff0-72be7b131ff7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:23.780196 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-ce92238b-cd43-40d0-8ff0-72be7b131ff7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:23.802429 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-ce92238b-cd43-40d0-8ff0-72be7b131ff7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1272 microversion: 2.1 time: 0.501779 Sep 04 01:05:23.803047 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 11/25] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:23 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1272 bytes in 503 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:23.945510 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-e4ec5e9b-1ba8-45f7-9846-d87164b75ee5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:23.946330 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-e4ec5e9b-1ba8-45f7-9846-d87164b75ee5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:24.077758 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e4ec5e9b-1ba8-45f7-9846-d87164b75ee5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:24.092940 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-e4ec5e9b-1ba8-45f7-9846-d87164b75ee5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1220 microversion: 2.1 time: 0.164730 Sep 04 01:05:24.100585 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 15/26] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:23 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1220 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:24.829857 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-d35a2397-9990-42e6-8962-7f77f41e44cd tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:24.831145 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-d35a2397-9990-42e6-8962-7f77f41e44cd tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:24.852417 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d35a2397-9990-42e6-8962-7f77f41e44cd tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:24.855346 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d35a2397-9990-42e6-8962-7f77f41e44cd tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:24.855743 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d35a2397-9990-42e6-8962-7f77f41e44cd tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:24.856474 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d35a2397-9990-42e6-8962-7f77f41e44cd tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:24.862122 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d35a2397-9990-42e6-8962-7f77f41e44cd tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:24.869871 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d35a2397-9990-42e6-8962-7f77f41e44cd tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:24.986911 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-2d67f70b-658d-4e3a-9363-6482da7a5f09 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-804255335", "imageRef": "a11315c3-ce81-478b-91e4-5353f189dfa7", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "1fb5aed8-696d-4f87-8b2a-7d29744538a8"}], "security_groups": [{"name": "tempest-securitygroup--98131319"}], "key_name": "tempest-keypair-677601389", "user_data": "}} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:05:25.496520 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.network.neutron [None req-2d67f70b-658d-4e3a-9363-6482da7a5f09 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] validate_networks() for [('1fb5aed8-696d-4f87-8b2a-7d29744538a8', None, None, None, None, None)] {{(pid=121036) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 04 01:05:26.365854 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-d35a2397-9990-42e6-8962-7f77f41e44cd tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:26.393793 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-d35a2397-9990-42e6-8962-7f77f41e44cd tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1314 microversion: 2.1 time: 1.557780 Sep 04 01:05:26.414243 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 12/27] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:24 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1314 bytes in 1589 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:26.430091 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-5dfb22e9-6936-4065-b2b5-ab42a2bd763d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:26.431542 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-5dfb22e9-6936-4065-b2b5-ab42a2bd763d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:26.450904 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5dfb22e9-6936-4065-b2b5-ab42a2bd763d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:26.455897 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5dfb22e9-6936-4065-b2b5-ab42a2bd763d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:26.455897 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5dfb22e9-6936-4065-b2b5-ab42a2bd763d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:26.469905 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5dfb22e9-6936-4065-b2b5-ab42a2bd763d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:26.470770 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5dfb22e9-6936-4065-b2b5-ab42a2bd763d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:26.471482 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5dfb22e9-6936-4065-b2b5-ab42a2bd763d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:27.113044 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-5dfb22e9-6936-4065-b2b5-ab42a2bd763d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:27.124765 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-5dfb22e9-6936-4065-b2b5-ab42a2bd763d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1272 microversion: 2.1 time: 0.708846 Sep 04 01:05:27.127532 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 13/28] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:26 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1272 bytes in 713 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:27.419556 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-6b5dc4a6-2e4c-4243-92a4-f93aea36b1d0 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:27.421036 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-6b5dc4a6-2e4c-4243-92a4-f93aea36b1d0 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:27.434971 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6b5dc4a6-2e4c-4243-92a4-f93aea36b1d0 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:27.438728 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6b5dc4a6-2e4c-4243-92a4-f93aea36b1d0 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:27.442147 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6b5dc4a6-2e4c-4243-92a4-f93aea36b1d0 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:27.443051 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6b5dc4a6-2e4c-4243-92a4-f93aea36b1d0 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:27.446618 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6b5dc4a6-2e4c-4243-92a4-f93aea36b1d0 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:27.448351 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6b5dc4a6-2e4c-4243-92a4-f93aea36b1d0 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:27.666441 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell0:SELECT=2 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:05:27.973221 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-2d67f70b-658d-4e3a-9363-6482da7a5f09 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Flavor limits 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 04 01:05:27.979776 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-2d67f70b-658d-4e3a-9363-6482da7a5f09 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Image limits 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 04 01:05:27.980609 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-2d67f70b-658d-4e3a-9363-6482da7a5f09 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Flavor pref 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 04 01:05:27.981204 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-2d67f70b-658d-4e3a-9363-6482da7a5f09 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Image pref 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 04 01:05:27.981632 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-2d67f70b-658d-4e3a-9363-6482da7a5f09 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 04 01:05:28.010181 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2d67f70b-658d-4e3a-9363-6482da7a5f09 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:28.010685 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2d67f70b-658d-4e3a-9363-6482da7a5f09 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:28.011130 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2d67f70b-658d-4e3a-9363-6482da7a5f09 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:28.011873 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2d67f70b-658d-4e3a-9363-6482da7a5f09 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:28.012553 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2d67f70b-658d-4e3a-9363-6482da7a5f09 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:28.013056 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2d67f70b-658d-4e3a-9363-6482da7a5f09 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:28.060092 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-2d67f70b-658d-4e3a-9363-6482da7a5f09 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Getting quotas for project e7dca68418a14d7da9aa63ac25f202e8. Resources: {'ram', 'instances', 'cores'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 04 01:05:28.071253 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-2d67f70b-658d-4e3a-9363-6482da7a5f09 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Getting quotas for user 44993cd93360442bb0afb28778ee83bc and project e7dca68418a14d7da9aa63ac25f202e8. Resources: {'ram', 'instances', 'cores'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 04 01:05:28.101773 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-2d67f70b-658d-4e3a-9363-6482da7a5f09 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Going to run 1 instances... {{(pid=121036) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 04 01:05:28.104554 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-2d67f70b-658d-4e3a-9363-6482da7a5f09 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a11315c3-ce81-478b-91e4-5353f189dfa7',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121036) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 04 01:05:28.296015 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-2d67f70b-658d-4e3a-9363-6482da7a5f09 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 400 microversion: 2.1 time: 3.318407 Sep 04 01:05:28.311312 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 16/29] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 4 01:05:24 2022] POST /compute/v2.1/servers => generated 400 bytes in 3338 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 04 01:05:28.339128 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-a1b2a27a-557a-4a41-8c16-d4da213608e6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:28.339788 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-a1b2a27a-557a-4a41-8c16-d4da213608e6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:28.372376 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a1b2a27a-557a-4a41-8c16-d4da213608e6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:28.375285 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a1b2a27a-557a-4a41-8c16-d4da213608e6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:28.377215 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a1b2a27a-557a-4a41-8c16-d4da213608e6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:28.381276 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a1b2a27a-557a-4a41-8c16-d4da213608e6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:28.382081 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a1b2a27a-557a-4a41-8c16-d4da213608e6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:28.383802 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a1b2a27a-557a-4a41-8c16-d4da213608e6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:28.434404 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-6b5dc4a6-2e4c-4243-92a4-f93aea36b1d0 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:28.474513 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-6b5dc4a6-2e4c-4243-92a4-f93aea36b1d0 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1314 microversion: 2.1 time: 1.059858 Sep 04 01:05:28.481226 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 14/30] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:27 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1314 bytes in 1067 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:28.498653 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-f0c9b6d7-cafe-494f-bde0-2a23129632eb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:28.502386 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-f0c9b6d7-cafe-494f-bde0-2a23129632eb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:28.575042 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-a1b2a27a-557a-4a41-8c16-d4da213608e6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:28.587375 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-a1b2a27a-557a-4a41-8c16-d4da213608e6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1278 microversion: 2.1 time: 0.274719 Sep 04 01:05:28.591377 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 17/31] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:28 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1278 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:28.617143 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-db76160e-12a1-4f01-bdf9-f27e0f48073b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1783337581", "imageRef": "a11315c3-ce81-478b-91e4-5353f189dfa7", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "d7f39628-666e-4480-8bee-49de183d5e09"}], "security_groups": [{"name": "tempest-securitygroup--1092877388"}], "key_name": "tempest-keypair-1366067967", "user_data": "}} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:05:28.915740 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-f0c9b6d7-cafe-494f-bde0-2a23129632eb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:28.953203 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-f0c9b6d7-cafe-494f-bde0-2a23129632eb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1236 microversion: 2.1 time: 0.470067 Sep 04 01:05:28.953721 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 15/32] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:28 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1236 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:29.509130 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-7b34f9b5-17fe-4e34-a09f-5a4cd8b9b347 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:29.509857 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-7b34f9b5-17fe-4e34-a09f-5a4cd8b9b347 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:29.529802 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7b34f9b5-17fe-4e34-a09f-5a4cd8b9b347 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:29.530330 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7b34f9b5-17fe-4e34-a09f-5a4cd8b9b347 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:29.530788 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7b34f9b5-17fe-4e34-a09f-5a4cd8b9b347 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:29.531527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7b34f9b5-17fe-4e34-a09f-5a4cd8b9b347 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:29.540521 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7b34f9b5-17fe-4e34-a09f-5a4cd8b9b347 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:29.543358 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7b34f9b5-17fe-4e34-a09f-5a4cd8b9b347 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:29.657458 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.network.neutron [None req-db76160e-12a1-4f01-bdf9-f27e0f48073b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] validate_networks() for [('d7f39628-666e-4480-8bee-49de183d5e09', None, None, None, None, None)] {{(pid=121036) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 04 01:05:30.645880 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-db76160e-12a1-4f01-bdf9-f27e0f48073b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Flavor limits 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 04 01:05:30.647114 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-db76160e-12a1-4f01-bdf9-f27e0f48073b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Image limits 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 04 01:05:30.647603 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-db76160e-12a1-4f01-bdf9-f27e0f48073b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Flavor pref 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 04 01:05:30.648187 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-db76160e-12a1-4f01-bdf9-f27e0f48073b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Image pref 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 04 01:05:30.649468 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-db76160e-12a1-4f01-bdf9-f27e0f48073b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 04 01:05:30.662145 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-db76160e-12a1-4f01-bdf9-f27e0f48073b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:30.663586 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-db76160e-12a1-4f01-bdf9-f27e0f48073b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:30.664194 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-db76160e-12a1-4f01-bdf9-f27e0f48073b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:30.665716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-db76160e-12a1-4f01-bdf9-f27e0f48073b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:30.666769 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-db76160e-12a1-4f01-bdf9-f27e0f48073b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:30.667363 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-db76160e-12a1-4f01-bdf9-f27e0f48073b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:30.690262 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-db76160e-12a1-4f01-bdf9-f27e0f48073b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Getting quotas for project 287d111f08ac4b05bd5adec9c40d0620. Resources: {'ram', 'instances', 'cores'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 04 01:05:30.695046 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-db76160e-12a1-4f01-bdf9-f27e0f48073b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Getting quotas for user f5e4723258c64ce5885154dded2d3d43 and project 287d111f08ac4b05bd5adec9c40d0620. Resources: {'ram', 'instances', 'cores'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 04 01:05:30.738570 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-db76160e-12a1-4f01-bdf9-f27e0f48073b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Going to run 1 instances... {{(pid=121036) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 04 01:05:30.747691 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-7b34f9b5-17fe-4e34-a09f-5a4cd8b9b347 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:30.752064 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-db76160e-12a1-4f01-bdf9-f27e0f48073b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a11315c3-ce81-478b-91e4-5353f189dfa7',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121036) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 04 01:05:30.761453 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-7b34f9b5-17fe-4e34-a09f-5a4cd8b9b347 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1314 microversion: 2.1 time: 1.255707 Sep 04 01:05:30.764410 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 16/33] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:29 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1314 bytes in 1262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:05:30.779841 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-58efb090-5ab4-4c34-81c7-a7989aa17a10 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1414057538", "imageRef": "a11315c3-ce81-478b-91e4-5353f189dfa7", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "3e702e6e-ebe1-46a7-b30d-f3d140ac2993"}], "security_groups": [{"name": "tempest-securitygroup--385291614"}], "key_name": "tempest-keypair-870969389", "user_data": "}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:05:31.017264 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-db76160e-12a1-4f01-bdf9-f27e0f48073b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 2.419245 Sep 04 01:05:31.020186 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 18/34] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 4 01:05:28 2022] POST /compute/v2.1/servers => generated 402 bytes in 2428 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 04 01:05:31.037135 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-664e3f63-cd6b-4b00-b351-9111923934da tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:31.042917 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-664e3f63-cd6b-4b00-b351-9111923934da tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:31.074231 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-664e3f63-cd6b-4b00-b351-9111923934da tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:31.074972 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-664e3f63-cd6b-4b00-b351-9111923934da tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:31.076112 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-664e3f63-cd6b-4b00-b351-9111923934da tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:31.080975 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-664e3f63-cd6b-4b00-b351-9111923934da tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:31.082501 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-664e3f63-cd6b-4b00-b351-9111923934da tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:31.083912 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-664e3f63-cd6b-4b00-b351-9111923934da tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:31.623735 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.network.neutron [None req-58efb090-5ab4-4c34-81c7-a7989aa17a10 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] validate_networks() for [('3e702e6e-ebe1-46a7-b30d-f3d140ac2993', None, None, None, None, None)] {{(pid=121037) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 04 01:05:32.141854 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell0:SELECT=2 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:05:32.246001 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-664e3f63-cd6b-4b00-b351-9111923934da tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:32.270701 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-664e3f63-cd6b-4b00-b351-9111923934da tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1320 microversion: 2.1 time: 1.244931 Sep 04 01:05:32.298522 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 19/35] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:31 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1320 bytes in 1277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:32.311398 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-c30f7eff-d906-41d1-bd88-245e78a48f10 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:32.326152 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-c30f7eff-d906-41d1-bd88-245e78a48f10 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:32.344640 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c30f7eff-d906-41d1-bd88-245e78a48f10 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:32.346093 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c30f7eff-d906-41d1-bd88-245e78a48f10 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:32.350249 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c30f7eff-d906-41d1-bd88-245e78a48f10 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:32.358183 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c30f7eff-d906-41d1-bd88-245e78a48f10 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:32.358183 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c30f7eff-d906-41d1-bd88-245e78a48f10 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:32.358183 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c30f7eff-d906-41d1-bd88-245e78a48f10 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:32.539891 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-c30f7eff-d906-41d1-bd88-245e78a48f10 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:32.555144 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-c30f7eff-d906-41d1-bd88-245e78a48f10 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1288 microversion: 2.1 time: 0.254912 Sep 04 01:05:32.565135 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 20/36] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:32 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1288 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:32.566220 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b3f0afbf-3750-4114-90d3-bef1a2d8d20a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:32.567136 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-b3f0afbf-3750-4114-90d3-bef1a2d8d20a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:32.570131 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_api:SELECT=90,nova_api:INSERT=10 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:05:32.703306 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-b3f0afbf-3750-4114-90d3-bef1a2d8d20a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:32.718111 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-b3f0afbf-3750-4114-90d3-bef1a2d8d20a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1242 microversion: 2.1 time: 0.160657 Sep 04 01:05:32.721286 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 21/37] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:32 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1242 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:32.854896 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-42808d81-4686-414e-bdd4-efafe086240b req-7d66b52e-5b6d-4f08-af58-eb19da21426e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "742cb4b8-f03c-4256-bb96-fc2cfff43734", "tag": "5d2b3e83-c7f4-4359-8491-abf0d6e3eec3"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:05:32.926712 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-42808d81-4686-414e-bdd4-efafe086240b req-7d66b52e-5b6d-4f08-af58-eb19da21426e service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:32.928990 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-42808d81-4686-414e-bdd4-efafe086240b req-7d66b52e-5b6d-4f08-af58-eb19da21426e service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:32.930434 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-42808d81-4686-414e-bdd4-efafe086240b req-7d66b52e-5b6d-4f08-af58-eb19da21426e service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:33.063719 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-42808d81-4686-414e-bdd4-efafe086240b req-7d66b52e-5b6d-4f08-af58-eb19da21426e service nova] Creating event network-changed:5d2b3e83-c7f4-4359-8491-abf0d6e3eec3 for instance 742cb4b8-f03c-4256-bb96-fc2cfff43734 on hv-cinder-maste Sep 04 01:05:33.063719 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-42808d81-4686-414e-bdd4-efafe086240b req-7d66b52e-5b6d-4f08-af58-eb19da21426e service nova] Acquiring lock "compute-rpcapi-router" {{(pid=121036) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:293}} Sep 04 01:05:33.063719 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-42808d81-4686-414e-bdd4-efafe086240b req-7d66b52e-5b6d-4f08-af58-eb19da21426e service nova] Acquired lock "compute-rpcapi-router" {{(pid=121036) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:296}} Sep 04 01:05:33.063719 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fb818ae0-0277-4109-a9c4-fa1f98a0d189 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:33.063719 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fb818ae0-0277-4109-a9c4-fa1f98a0d189 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:33.064787 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fb818ae0-0277-4109-a9c4-fa1f98a0d189 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:33.064787 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fb818ae0-0277-4109-a9c4-fa1f98a0d189 None None] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:33.064787 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fb818ae0-0277-4109-a9c4-fa1f98a0d189 None None] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:33.064787 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fb818ae0-0277-4109-a9c4-fa1f98a0d189 None None] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:33.064787 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.compute.rpcapi [None req-fb818ae0-0277-4109-a9c4-fa1f98a0d189 None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Sep 04 01:05:33.067356 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fb818ae0-0277-4109-a9c4-fa1f98a0d189 None None] Releasing lock "compute-rpcapi-router" {{(pid=121036) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:314}} Sep 04 01:05:33.067356 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-fb818ae0-0277-4109-a9c4-fa1f98a0d189 None None] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.317316 Sep 04 01:05:33.067356 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 22/38] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:05:32 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:05:33.073134 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-27361660-2274-42c8-84e2-aa25d6c6e985 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:33.073915 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-27361660-2274-42c8-84e2-aa25d6c6e985 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:33.092897 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-27361660-2274-42c8-84e2-aa25d6c6e985 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:33.093441 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-27361660-2274-42c8-84e2-aa25d6c6e985 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:33.093901 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-27361660-2274-42c8-84e2-aa25d6c6e985 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:33.101196 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-27361660-2274-42c8-84e2-aa25d6c6e985 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:33.102531 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-27361660-2274-42c8-84e2-aa25d6c6e985 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:33.103614 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-27361660-2274-42c8-84e2-aa25d6c6e985 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:33.594010 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-27361660-2274-42c8-84e2-aa25d6c6e985 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:33.611589 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-27361660-2274-42c8-84e2-aa25d6c6e985 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1314 microversion: 2.1 time: 0.547724 Sep 04 01:05:33.629169 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 23/39] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:33 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1314 bytes in 556 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:33.629169 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-2a6eb2f7-9ab1-45e9-9e92-e3888e383c3c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:33.642603 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-2a6eb2f7-9ab1-45e9-9e92-e3888e383c3c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:33.663827 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-58efb090-5ab4-4c34-81c7-a7989aa17a10 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Flavor limits 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 04 01:05:33.673498 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-58efb090-5ab4-4c34-81c7-a7989aa17a10 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Image limits 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 04 01:05:33.675470 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2a6eb2f7-9ab1-45e9-9e92-e3888e383c3c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:33.676111 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2a6eb2f7-9ab1-45e9-9e92-e3888e383c3c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.024s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:33.676562 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2a6eb2f7-9ab1-45e9-9e92-e3888e383c3c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:33.679819 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2a6eb2f7-9ab1-45e9-9e92-e3888e383c3c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:33.680320 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2a6eb2f7-9ab1-45e9-9e92-e3888e383c3c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:33.680761 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2a6eb2f7-9ab1-45e9-9e92-e3888e383c3c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:33.681156 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-58efb090-5ab4-4c34-81c7-a7989aa17a10 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Flavor pref 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 04 01:05:33.681560 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-58efb090-5ab4-4c34-81c7-a7989aa17a10 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Image pref 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 04 01:05:33.687762 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-58efb090-5ab4-4c34-81c7-a7989aa17a10 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 04 01:05:33.725664 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-58efb090-5ab4-4c34-81c7-a7989aa17a10 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:33.726167 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-58efb090-5ab4-4c34-81c7-a7989aa17a10 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:33.726614 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-58efb090-5ab4-4c34-81c7-a7989aa17a10 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:33.727354 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-58efb090-5ab4-4c34-81c7-a7989aa17a10 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:33.727816 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-58efb090-5ab4-4c34-81c7-a7989aa17a10 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:33.731640 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-58efb090-5ab4-4c34-81c7-a7989aa17a10 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:33.786281 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.quota [None req-58efb090-5ab4-4c34-81c7-a7989aa17a10 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Getting quotas for project 55faed42b7f243df9df879d581797f56. Resources: {'ram', 'instances', 'cores'} {{(pid=121037) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 04 01:05:33.790378 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.quota [None req-58efb090-5ab4-4c34-81c7-a7989aa17a10 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Getting quotas for user fbcd4c19e766446188e3b3f41a44e2e4 and project 55faed42b7f243df9df879d581797f56. Resources: {'ram', 'instances', 'cores'} {{(pid=121037) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 04 01:05:33.807041 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-58efb090-5ab4-4c34-81c7-a7989aa17a10 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Going to run 1 instances... {{(pid=121037) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 04 01:05:33.810633 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-58efb090-5ab4-4c34-81c7-a7989aa17a10 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a11315c3-ce81-478b-91e4-5353f189dfa7',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121037) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 04 01:05:34.026993 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-58efb090-5ab4-4c34-81c7-a7989aa17a10 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.20 time: 3.257440 Sep 04 01:05:34.028463 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 17/40] 10.222.0.42 () {62 vars in 1188 bytes} [Sun Sep 4 01:05:30 2022] POST /compute/v2.1/servers => generated 401 bytes in 3263 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Sep 04 01:05:34.045931 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-dad619a2-ba79-4a99-82eb-ecad21c7cf84 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:34.047925 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-dad619a2-ba79-4a99-82eb-ecad21c7cf84 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:34.081744 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-dad619a2-ba79-4a99-82eb-ecad21c7cf84 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:34.084924 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-dad619a2-ba79-4a99-82eb-ecad21c7cf84 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:34.086699 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-dad619a2-ba79-4a99-82eb-ecad21c7cf84 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:34.089999 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-dad619a2-ba79-4a99-82eb-ecad21c7cf84 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:34.093113 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-dad619a2-ba79-4a99-82eb-ecad21c7cf84 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:34.097971 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-dad619a2-ba79-4a99-82eb-ecad21c7cf84 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:34.286744 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-2a6eb2f7-9ab1-45e9-9e92-e3888e383c3c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:34.302533 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-2a6eb2f7-9ab1-45e9-9e92-e3888e383c3c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1320 microversion: 2.1 time: 0.681851 Sep 04 01:05:34.306567 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 24/41] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:33 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1320 bytes in 683 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:34.331873 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-485402d7-c3df-4384-9bff-4f24dc2425f9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:34.332777 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-485402d7-c3df-4384-9bff-4f24dc2425f9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:34.387803 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-dad619a2-ba79-4a99-82eb-ecad21c7cf84 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:34.402534 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-485402d7-c3df-4384-9bff-4f24dc2425f9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:34.403872 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-485402d7-c3df-4384-9bff-4f24dc2425f9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:34.405893 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-485402d7-c3df-4384-9bff-4f24dc2425f9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:34.407088 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-485402d7-c3df-4384-9bff-4f24dc2425f9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:34.408589 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-485402d7-c3df-4384-9bff-4f24dc2425f9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:34.413019 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-485402d7-c3df-4384-9bff-4f24dc2425f9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:34.418800 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-dad619a2-ba79-4a99-82eb-ecad21c7cf84 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1288 microversion: 2.1 time: 0.378480 Sep 04 01:05:34.419400 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 18/42] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:34 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1288 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:34.433235 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-a4752a27-ef03-42bc-bcba-787dc5916925 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:34.433886 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-a4752a27-ef03-42bc-bcba-787dc5916925 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:34.476159 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_api:SELECT=89,nova_api:INSERT=11 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:05:34.647006 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-a4752a27-ef03-42bc-bcba-787dc5916925 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:34.659904 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-a4752a27-ef03-42bc-bcba-787dc5916925 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:34.667024 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-a4752a27-ef03-42bc-bcba-787dc5916925 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:34.675997 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-a4752a27-ef03-42bc-bcba-787dc5916925 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1281 microversion: 2.20 time: 0.251000 Sep 04 01:05:34.702159 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 19/43] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:05:34 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1281 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:05:34.726840 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-4d079f17-fe1e-4bb7-87d3-71d003615f52 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:34.729194 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-4d079f17-fe1e-4bb7-87d3-71d003615f52 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:34.755499 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4d079f17-fe1e-4bb7-87d3-71d003615f52 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:34.756180 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4d079f17-fe1e-4bb7-87d3-71d003615f52 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:34.756708 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4d079f17-fe1e-4bb7-87d3-71d003615f52 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:34.759895 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4d079f17-fe1e-4bb7-87d3-71d003615f52 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:34.760726 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4d079f17-fe1e-4bb7-87d3-71d003615f52 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:34.763833 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4d079f17-fe1e-4bb7-87d3-71d003615f52 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:35.079694 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-4d079f17-fe1e-4bb7-87d3-71d003615f52 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:35.095677 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-4d079f17-fe1e-4bb7-87d3-71d003615f52 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1314 microversion: 2.1 time: 0.391238 Sep 04 01:05:35.109109 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 20/44] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:34 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1314 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:35.239830 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-485402d7-c3df-4384-9bff-4f24dc2425f9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:35.259412 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-485402d7-c3df-4384-9bff-4f24dc2425f9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1246 microversion: 2.1 time: 0.935752 Sep 04 01:05:35.260447 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 25/45] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:34 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1246 bytes in 957 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:35.327931 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-5f4e7153-10f2-48fb-9a0d-da6b7f768dad tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:35.328537 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-5f4e7153-10f2-48fb-9a0d-da6b7f768dad tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:35.335434 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5f4e7153-10f2-48fb-9a0d-da6b7f768dad tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:35.335947 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5f4e7153-10f2-48fb-9a0d-da6b7f768dad tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:35.336382 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5f4e7153-10f2-48fb-9a0d-da6b7f768dad tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:35.337229 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5f4e7153-10f2-48fb-9a0d-da6b7f768dad tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:35.337669 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5f4e7153-10f2-48fb-9a0d-da6b7f768dad tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:35.338100 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5f4e7153-10f2-48fb-9a0d-da6b7f768dad tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:35.367955 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-f252461e-7abc-42ab-b04d-8ab1ec8a0811 req-d2e9f3c5-8875-45a9-83f7-ac78b9679410 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "62c4a968-ecfa-4842-8cab-8aaeb7ca7a64", "tag": "addc777a-4bdc-4153-92fa-8184c15d2cd1"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:05:35.404901 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-f252461e-7abc-42ab-b04d-8ab1ec8a0811 req-d2e9f3c5-8875-45a9-83f7-ac78b9679410 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:35.405993 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-f252461e-7abc-42ab-b04d-8ab1ec8a0811 req-d2e9f3c5-8875-45a9-83f7-ac78b9679410 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:35.406678 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-f252461e-7abc-42ab-b04d-8ab1ec8a0811 req-d2e9f3c5-8875-45a9-83f7-ac78b9679410 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:35.473917 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-f252461e-7abc-42ab-b04d-8ab1ec8a0811 req-d2e9f3c5-8875-45a9-83f7-ac78b9679410 service nova] Creating event network-changed:addc777a-4bdc-4153-92fa-8184c15d2cd1 for instance 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 on hv-cinder-maste Sep 04 01:05:35.474458 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-f252461e-7abc-42ab-b04d-8ab1ec8a0811 req-d2e9f3c5-8875-45a9-83f7-ac78b9679410 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=121037) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:293}} Sep 04 01:05:35.474853 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-f252461e-7abc-42ab-b04d-8ab1ec8a0811 req-d2e9f3c5-8875-45a9-83f7-ac78b9679410 service nova] Acquired lock "compute-rpcapi-router" {{(pid=121037) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:296}} Sep 04 01:05:35.479609 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e9f0a21b-e8c6-44b0-8e31-906b56e6dba4 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:35.480095 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e9f0a21b-e8c6-44b0-8e31-906b56e6dba4 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:35.480536 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e9f0a21b-e8c6-44b0-8e31-906b56e6dba4 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:35.481305 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e9f0a21b-e8c6-44b0-8e31-906b56e6dba4 None None] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:35.481772 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e9f0a21b-e8c6-44b0-8e31-906b56e6dba4 None None] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:35.486415 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e9f0a21b-e8c6-44b0-8e31-906b56e6dba4 None None] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:35.507976 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.compute.rpcapi [None req-e9f0a21b-e8c6-44b0-8e31-906b56e6dba4 None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Sep 04 01:05:35.509166 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e9f0a21b-e8c6-44b0-8e31-906b56e6dba4 None None] Releasing lock "compute-rpcapi-router" {{(pid=121037) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:314}} Sep 04 01:05:35.559971 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-e9f0a21b-e8c6-44b0-8e31-906b56e6dba4 None None] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.361169 Sep 04 01:05:35.563826 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 21/46] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:05:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:05:35.568435 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-738de888-1d97-4a9b-bb23-d6943e1caae6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:35.569404 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-738de888-1d97-4a9b-bb23-d6943e1caae6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:35.578424 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-738de888-1d97-4a9b-bb23-d6943e1caae6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:35.579156 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-738de888-1d97-4a9b-bb23-d6943e1caae6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:35.579751 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-738de888-1d97-4a9b-bb23-d6943e1caae6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:35.584961 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-738de888-1d97-4a9b-bb23-d6943e1caae6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:35.585799 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-738de888-1d97-4a9b-bb23-d6943e1caae6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:35.586444 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-738de888-1d97-4a9b-bb23-d6943e1caae6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:35.967588 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-738de888-1d97-4a9b-bb23-d6943e1caae6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:35.985855 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-738de888-1d97-4a9b-bb23-d6943e1caae6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1306 microversion: 2.1 time: 0.423400 Sep 04 01:05:35.987054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 22/47] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:35 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1306 bytes in 426 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:35.992307 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-3d55bb13-0b83-4723-a304-6a015b2ab248 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:35.993414 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-3d55bb13-0b83-4723-a304-6a015b2ab248 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:36.013934 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-5f4e7153-10f2-48fb-9a0d-da6b7f768dad tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:36.030989 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-5f4e7153-10f2-48fb-9a0d-da6b7f768dad tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1320 microversion: 2.1 time: 0.706681 Sep 04 01:05:36.032160 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 26/48] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:35 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1320 bytes in 709 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:05:36.042038 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-edb9c353-27a4-4fed-9b5b-6575ac1bda7d req-36670936-2dfd-41ca-a3a8-5d88b2f6662a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2f7a7e72-1f16-4ab4-b826-d376d39be4c2", "tag": "28efc83e-9838-45a7-9fd3-3625eeff4b09"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:05:36.070137 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-edb9c353-27a4-4fed-9b5b-6575ac1bda7d req-36670936-2dfd-41ca-a3a8-5d88b2f6662a service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:36.070537 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-edb9c353-27a4-4fed-9b5b-6575ac1bda7d req-36670936-2dfd-41ca-a3a8-5d88b2f6662a service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:36.070982 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-edb9c353-27a4-4fed-9b5b-6575ac1bda7d req-36670936-2dfd-41ca-a3a8-5d88b2f6662a service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:36.105774 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-edb9c353-27a4-4fed-9b5b-6575ac1bda7d req-36670936-2dfd-41ca-a3a8-5d88b2f6662a service nova] Creating event network-changed:28efc83e-9838-45a7-9fd3-3625eeff4b09 for instance 2f7a7e72-1f16-4ab4-b826-d376d39be4c2 on hv-cinder-maste Sep 04 01:05:36.122324 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-edb9c353-27a4-4fed-9b5b-6575ac1bda7d req-36670936-2dfd-41ca-a3a8-5d88b2f6662a service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.087192 Sep 04 01:05:36.123451 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 27/49] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:05:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:05:36.129861 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-c8766c04-55f0-4a69-9491-d7823d873b28 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:36.139170 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-c8766c04-55f0-4a69-9491-d7823d873b28 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:36.149013 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-3d55bb13-0b83-4723-a304-6a015b2ab248 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:36.164013 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-3d55bb13-0b83-4723-a304-6a015b2ab248 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:36.164116 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c8766c04-55f0-4a69-9491-d7823d873b28 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:36.164201 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-3d55bb13-0b83-4723-a304-6a015b2ab248 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:36.165204 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-3d55bb13-0b83-4723-a304-6a015b2ab248 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1281 microversion: 2.20 time: 0.177075 Sep 04 01:05:36.165988 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 23/50] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:05:35 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1281 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:05:36.177446 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c8766c04-55f0-4a69-9491-d7823d873b28 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:36.179454 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c8766c04-55f0-4a69-9491-d7823d873b28 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.017s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:36.183182 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c8766c04-55f0-4a69-9491-d7823d873b28 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:36.184211 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c8766c04-55f0-4a69-9491-d7823d873b28 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:36.188133 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c8766c04-55f0-4a69-9491-d7823d873b28 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:36.292381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-7edeaa29-43e5-4878-a96a-e4d2d1647995 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:36.293289 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-7edeaa29-43e5-4878-a96a-e4d2d1647995 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:36.309117 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7edeaa29-43e5-4878-a96a-e4d2d1647995 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:36.310526 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7edeaa29-43e5-4878-a96a-e4d2d1647995 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:36.313186 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7edeaa29-43e5-4878-a96a-e4d2d1647995 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:36.314086 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7edeaa29-43e5-4878-a96a-e4d2d1647995 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:36.316190 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7edeaa29-43e5-4878-a96a-e4d2d1647995 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:36.318272 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7edeaa29-43e5-4878-a96a-e4d2d1647995 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:36.534115 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-7edeaa29-43e5-4878-a96a-e4d2d1647995 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:36.545902 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-7edeaa29-43e5-4878-a96a-e4d2d1647995 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1294 microversion: 2.1 time: 0.259916 Sep 04 01:05:36.548086 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 24/51] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:36 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1294 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:36.719710 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-c8766c04-55f0-4a69-9491-d7823d873b28 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:36.719710 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-c8766c04-55f0-4a69-9491-d7823d873b28 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1314 microversion: 2.1 time: 0.585590 Sep 04 01:05:36.719710 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 28/52] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:36 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1314 bytes in 587 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:37.018151 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-f7821844-f4fe-43aa-b581-b69ae04144d9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:37.019708 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-f7821844-f4fe-43aa-b581-b69ae04144d9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:37.032249 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f7821844-f4fe-43aa-b581-b69ae04144d9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:37.032744 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f7821844-f4fe-43aa-b581-b69ae04144d9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:37.033213 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f7821844-f4fe-43aa-b581-b69ae04144d9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:37.034748 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f7821844-f4fe-43aa-b581-b69ae04144d9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:37.034965 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f7821844-f4fe-43aa-b581-b69ae04144d9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:37.035315 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f7821844-f4fe-43aa-b581-b69ae04144d9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:37.069047 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-377e2ee7-040f-4646-ae7a-5c7321f17b14 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:37.071519 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-377e2ee7-040f-4646-ae7a-5c7321f17b14 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:37.086742 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-377e2ee7-040f-4646-ae7a-5c7321f17b14 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:37.087391 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-377e2ee7-040f-4646-ae7a-5c7321f17b14 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:37.087904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-377e2ee7-040f-4646-ae7a-5c7321f17b14 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:37.088718 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-377e2ee7-040f-4646-ae7a-5c7321f17b14 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:37.089294 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-377e2ee7-040f-4646-ae7a-5c7321f17b14 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:37.090236 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-377e2ee7-040f-4646-ae7a-5c7321f17b14 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:37.124343 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-f7821844-f4fe-43aa-b581-b69ae04144d9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:37.132799 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-f7821844-f4fe-43aa-b581-b69ae04144d9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1294 microversion: 2.1 time: 0.120474 Sep 04 01:05:37.133581 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 25/53] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:37 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1294 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:37.194984 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-2826bfe6-2053-4d29-82cc-ead472bf25b3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:37.195658 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-2826bfe6-2053-4d29-82cc-ead472bf25b3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:37.201701 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2826bfe6-2053-4d29-82cc-ead472bf25b3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:37.202175 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2826bfe6-2053-4d29-82cc-ead472bf25b3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:37.202766 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2826bfe6-2053-4d29-82cc-ead472bf25b3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:37.203606 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2826bfe6-2053-4d29-82cc-ead472bf25b3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:37.204280 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2826bfe6-2053-4d29-82cc-ead472bf25b3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:37.204936 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2826bfe6-2053-4d29-82cc-ead472bf25b3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:37.306083 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-2826bfe6-2053-4d29-82cc-ead472bf25b3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:37.314666 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-2826bfe6-2053-4d29-82cc-ead472bf25b3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:37.318626 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-2826bfe6-2053-4d29-82cc-ead472bf25b3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:37.318626 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-2826bfe6-2053-4d29-82cc-ead472bf25b3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1285 microversion: 2.20 time: 0.126264 Sep 04 01:05:37.333516 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-377e2ee7-040f-4646-ae7a-5c7321f17b14 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:37.335859 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 26/54] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:05:37 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1285 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:05:37.341527 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-377e2ee7-040f-4646-ae7a-5c7321f17b14 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1320 microversion: 2.1 time: 0.290340 Sep 04 01:05:37.352944 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 29/55] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:37 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1320 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:05:37.571266 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-99af3b73-2ac6-4cfd-a3c8-d27cb32df5c1 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:37.571999 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-99af3b73-2ac6-4cfd-a3c8-d27cb32df5c1 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:37.577997 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-99af3b73-2ac6-4cfd-a3c8-d27cb32df5c1 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:37.581419 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-99af3b73-2ac6-4cfd-a3c8-d27cb32df5c1 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:37.581419 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-99af3b73-2ac6-4cfd-a3c8-d27cb32df5c1 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:37.581419 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-99af3b73-2ac6-4cfd-a3c8-d27cb32df5c1 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:37.581419 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-99af3b73-2ac6-4cfd-a3c8-d27cb32df5c1 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:37.581951 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-99af3b73-2ac6-4cfd-a3c8-d27cb32df5c1 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:37.666544 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-99af3b73-2ac6-4cfd-a3c8-d27cb32df5c1 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:37.676423 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-99af3b73-2ac6-4cfd-a3c8-d27cb32df5c1 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1294 microversion: 2.1 time: 0.109145 Sep 04 01:05:37.685652 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 27/56] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:37 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1294 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:37.748108 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-a7c40c3a-fe08-4c2e-a2f9-42c481f6ac2a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:37.748726 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-a7c40c3a-fe08-4c2e-a2f9-42c481f6ac2a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:37.755716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7c40c3a-fe08-4c2e-a2f9-42c481f6ac2a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:37.756133 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7c40c3a-fe08-4c2e-a2f9-42c481f6ac2a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:37.756735 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7c40c3a-fe08-4c2e-a2f9-42c481f6ac2a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:37.757694 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7c40c3a-fe08-4c2e-a2f9-42c481f6ac2a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:37.758142 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7c40c3a-fe08-4c2e-a2f9-42c481f6ac2a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:37.758566 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7c40c3a-fe08-4c2e-a2f9-42c481f6ac2a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:37.905458 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-a7c40c3a-fe08-4c2e-a2f9-42c481f6ac2a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:37.915314 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-a7c40c3a-fe08-4c2e-a2f9-42c481f6ac2a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1314 microversion: 2.1 time: 0.169687 Sep 04 01:05:37.915314 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 30/57] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:37 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1314 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:38.160570 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-f900ecb2-61f3-4a95-b9b3-6da8a14d0354 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:38.161218 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-f900ecb2-61f3-4a95-b9b3-6da8a14d0354 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:38.168464 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f900ecb2-61f3-4a95-b9b3-6da8a14d0354 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:38.169120 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f900ecb2-61f3-4a95-b9b3-6da8a14d0354 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:38.169418 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f900ecb2-61f3-4a95-b9b3-6da8a14d0354 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:38.173592 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f900ecb2-61f3-4a95-b9b3-6da8a14d0354 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:38.173953 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f900ecb2-61f3-4a95-b9b3-6da8a14d0354 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:38.175205 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f900ecb2-61f3-4a95-b9b3-6da8a14d0354 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:38.313795 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-f900ecb2-61f3-4a95-b9b3-6da8a14d0354 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:38.320560 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=93 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:05:38.323097 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-f900ecb2-61f3-4a95-b9b3-6da8a14d0354 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1360 microversion: 2.1 time: 0.167881 Sep 04 01:05:38.323672 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 28/58] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:38 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1360 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:38.347203 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-c0916b10-f9dd-4f96-814b-9dcb11c9df0d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:38.348319 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-c0916b10-f9dd-4f96-814b-9dcb11c9df0d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:38.354274 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c0916b10-f9dd-4f96-814b-9dcb11c9df0d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:38.354762 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c0916b10-f9dd-4f96-814b-9dcb11c9df0d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:38.355201 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c0916b10-f9dd-4f96-814b-9dcb11c9df0d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:38.355894 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c0916b10-f9dd-4f96-814b-9dcb11c9df0d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:38.356474 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c0916b10-f9dd-4f96-814b-9dcb11c9df0d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:38.356906 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c0916b10-f9dd-4f96-814b-9dcb11c9df0d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:38.367836 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-a60b5247-8859-44bf-a63e-c8627cc844b5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:38.372495 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-a60b5247-8859-44bf-a63e-c8627cc844b5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:38.374782 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a60b5247-8859-44bf-a63e-c8627cc844b5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:38.375454 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a60b5247-8859-44bf-a63e-c8627cc844b5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:38.375897 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a60b5247-8859-44bf-a63e-c8627cc844b5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:38.376730 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a60b5247-8859-44bf-a63e-c8627cc844b5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:38.377363 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a60b5247-8859-44bf-a63e-c8627cc844b5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:38.378178 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a60b5247-8859-44bf-a63e-c8627cc844b5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:38.379983 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=100 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:05:38.427062 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-c0916b10-f9dd-4f96-814b-9dcb11c9df0d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:38.440556 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-c0916b10-f9dd-4f96-814b-9dcb11c9df0d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:38.442912 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-c0916b10-f9dd-4f96-814b-9dcb11c9df0d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:38.444651 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-c0916b10-f9dd-4f96-814b-9dcb11c9df0d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1285 microversion: 2.20 time: 0.102869 Sep 04 01:05:38.453492 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 31/59] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:05:38 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1285 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:05:38.523066 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-a60b5247-8859-44bf-a63e-c8627cc844b5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:38.530937 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-a60b5247-8859-44bf-a63e-c8627cc844b5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1320 microversion: 2.1 time: 0.167812 Sep 04 01:05:38.531502 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 29/60] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:38 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1320 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:38.699813 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-9ae44b37-d65d-4eeb-8c9a-c8b3c32367bd tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:38.700408 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-9ae44b37-d65d-4eeb-8c9a-c8b3c32367bd tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:38.707426 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9ae44b37-d65d-4eeb-8c9a-c8b3c32367bd tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:38.707946 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9ae44b37-d65d-4eeb-8c9a-c8b3c32367bd tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:38.708387 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9ae44b37-d65d-4eeb-8c9a-c8b3c32367bd tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:38.709093 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9ae44b37-d65d-4eeb-8c9a-c8b3c32367bd tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:38.709952 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9ae44b37-d65d-4eeb-8c9a-c8b3c32367bd tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:38.710641 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9ae44b37-d65d-4eeb-8c9a-c8b3c32367bd tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:38.775764 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-9ae44b37-d65d-4eeb-8c9a-c8b3c32367bd tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:38.783286 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-9ae44b37-d65d-4eeb-8c9a-c8b3c32367bd tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1294 microversion: 2.1 time: 0.087357 Sep 04 01:05:38.784027 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 32/61] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:38 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1294 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:38.939818 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-b02dce82-3ae5-43b1-8044-caad978a184b tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:38.941295 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-b02dce82-3ae5-43b1-8044-caad978a184b tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:38.950510 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b02dce82-3ae5-43b1-8044-caad978a184b tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:38.952234 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b02dce82-3ae5-43b1-8044-caad978a184b tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:38.952234 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b02dce82-3ae5-43b1-8044-caad978a184b tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:38.952234 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b02dce82-3ae5-43b1-8044-caad978a184b tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:38.952586 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b02dce82-3ae5-43b1-8044-caad978a184b tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:38.953006 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b02dce82-3ae5-43b1-8044-caad978a184b tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:39.093571 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-b02dce82-3ae5-43b1-8044-caad978a184b tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:39.102877 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-b02dce82-3ae5-43b1-8044-caad978a184b tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1314 microversion: 2.1 time: 0.166735 Sep 04 01:05:39.103648 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 30/62] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:38 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1314 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:39.354703 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-e6a76074-f888-4334-a94c-9aa921ce7f17 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:39.356726 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-e6a76074-f888-4334-a94c-9aa921ce7f17 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:39.364123 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e6a76074-f888-4334-a94c-9aa921ce7f17 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:39.364613 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e6a76074-f888-4334-a94c-9aa921ce7f17 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:39.365134 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e6a76074-f888-4334-a94c-9aa921ce7f17 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:39.365839 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e6a76074-f888-4334-a94c-9aa921ce7f17 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:39.366343 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e6a76074-f888-4334-a94c-9aa921ce7f17 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:39.366775 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e6a76074-f888-4334-a94c-9aa921ce7f17 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:39.470950 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-10c41491-2499-41ca-8897-7c77553e7eb8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:39.473901 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-10c41491-2499-41ca-8897-7c77553e7eb8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:39.482820 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-10c41491-2499-41ca-8897-7c77553e7eb8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:39.483308 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-10c41491-2499-41ca-8897-7c77553e7eb8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:39.483926 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-10c41491-2499-41ca-8897-7c77553e7eb8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:39.484637 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-10c41491-2499-41ca-8897-7c77553e7eb8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:39.485208 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-10c41491-2499-41ca-8897-7c77553e7eb8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:39.485698 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-10c41491-2499-41ca-8897-7c77553e7eb8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:39.530290 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e6a76074-f888-4334-a94c-9aa921ce7f17 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:39.540469 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-e6a76074-f888-4334-a94c-9aa921ce7f17 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1360 microversion: 2.1 time: 0.190412 Sep 04 01:05:39.541510 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 33/63] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:39 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1360 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:39.565424 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-7ee0281d-7afe-4e7a-b8ab-91e06a36ddc2 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:39.566880 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-7ee0281d-7afe-4e7a-b8ab-91e06a36ddc2 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:39.578041 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7ee0281d-7afe-4e7a-b8ab-91e06a36ddc2 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:39.578873 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7ee0281d-7afe-4e7a-b8ab-91e06a36ddc2 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:39.579762 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7ee0281d-7afe-4e7a-b8ab-91e06a36ddc2 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:39.581251 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7ee0281d-7afe-4e7a-b8ab-91e06a36ddc2 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:39.581908 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7ee0281d-7afe-4e7a-b8ab-91e06a36ddc2 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:39.582540 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7ee0281d-7afe-4e7a-b8ab-91e06a36ddc2 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:39.591128 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-10c41491-2499-41ca-8897-7c77553e7eb8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:39.600741 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-10c41491-2499-41ca-8897-7c77553e7eb8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:39.602255 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-10c41491-2499-41ca-8897-7c77553e7eb8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:39.603747 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-10c41491-2499-41ca-8897-7c77553e7eb8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1285 microversion: 2.20 time: 0.137515 Sep 04 01:05:39.606623 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 31/64] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:05:39 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1285 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:05:39.725362 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-7ee0281d-7afe-4e7a-b8ab-91e06a36ddc2 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:39.733109 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-7ee0281d-7afe-4e7a-b8ab-91e06a36ddc2 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1320 microversion: 2.1 time: 0.179430 Sep 04 01:05:39.736638 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 34/65] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:39 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1320 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:39.805983 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-9d073f2c-c92b-43be-8986-671e005bd797 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:39.806609 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-9d073f2c-c92b-43be-8986-671e005bd797 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:39.812991 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9d073f2c-c92b-43be-8986-671e005bd797 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:39.814034 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9d073f2c-c92b-43be-8986-671e005bd797 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:39.814476 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9d073f2c-c92b-43be-8986-671e005bd797 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:39.815210 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9d073f2c-c92b-43be-8986-671e005bd797 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:39.815763 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9d073f2c-c92b-43be-8986-671e005bd797 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:39.816302 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9d073f2c-c92b-43be-8986-671e005bd797 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:39.891904 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-9d073f2c-c92b-43be-8986-671e005bd797 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:39.902497 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-9d073f2c-c92b-43be-8986-671e005bd797 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1294 microversion: 2.1 time: 0.100165 Sep 04 01:05:39.903076 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 32/66] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:39 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1294 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:40.132470 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-7552866d-8406-47ec-87c8-8b861e1c81b8 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:40.132470 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-7552866d-8406-47ec-87c8-8b861e1c81b8 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:40.137776 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7552866d-8406-47ec-87c8-8b861e1c81b8 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:40.138586 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7552866d-8406-47ec-87c8-8b861e1c81b8 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:40.139415 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7552866d-8406-47ec-87c8-8b861e1c81b8 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:40.140344 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7552866d-8406-47ec-87c8-8b861e1c81b8 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:40.141464 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7552866d-8406-47ec-87c8-8b861e1c81b8 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:40.141903 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7552866d-8406-47ec-87c8-8b861e1c81b8 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:40.592775 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-9b7eb35d-ca4e-4344-8b9f-97f3be870a5b tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:40.594043 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-9b7eb35d-ca4e-4344-8b9f-97f3be870a5b tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:40.604564 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9b7eb35d-ca4e-4344-8b9f-97f3be870a5b tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:40.605052 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9b7eb35d-ca4e-4344-8b9f-97f3be870a5b tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:40.605836 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9b7eb35d-ca4e-4344-8b9f-97f3be870a5b tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:40.606724 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9b7eb35d-ca4e-4344-8b9f-97f3be870a5b tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:40.607359 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9b7eb35d-ca4e-4344-8b9f-97f3be870a5b tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:40.608723 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9b7eb35d-ca4e-4344-8b9f-97f3be870a5b tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:40.670700 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-7552866d-8406-47ec-87c8-8b861e1c81b8 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:40.681519 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-7552866d-8406-47ec-87c8-8b861e1c81b8 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1314 microversion: 2.1 time: 0.556422 Sep 04 01:05:40.682117 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 35/67] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:40 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1314 bytes in 558 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:40.694888 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-97a20e3e-7a70-45f2-bfea-72e4afc95fe6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:40.695774 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-97a20e3e-7a70-45f2-bfea-72e4afc95fe6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:40.702885 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-97a20e3e-7a70-45f2-bfea-72e4afc95fe6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:40.703498 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-97a20e3e-7a70-45f2-bfea-72e4afc95fe6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:40.704034 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-97a20e3e-7a70-45f2-bfea-72e4afc95fe6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:40.704916 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-97a20e3e-7a70-45f2-bfea-72e4afc95fe6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:40.705560 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-97a20e3e-7a70-45f2-bfea-72e4afc95fe6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:40.706763 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-97a20e3e-7a70-45f2-bfea-72e4afc95fe6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:40.777890 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-9b7eb35d-ca4e-4344-8b9f-97f3be870a5b tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:40.787598 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-9b7eb35d-ca4e-4344-8b9f-97f3be870a5b tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1360 microversion: 2.1 time: 0.201755 Sep 04 01:05:40.789362 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 33/68] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:40 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1360 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:40.794400 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-62337368-0d27-40b0-a470-2b7ff0f55583 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:40.795236 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-62337368-0d27-40b0-a470-2b7ff0f55583 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:40.798480 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-97a20e3e-7a70-45f2-bfea-72e4afc95fe6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:40.802143 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-62337368-0d27-40b0-a470-2b7ff0f55583 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:40.803134 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-62337368-0d27-40b0-a470-2b7ff0f55583 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:40.804007 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-62337368-0d27-40b0-a470-2b7ff0f55583 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:40.804885 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-62337368-0d27-40b0-a470-2b7ff0f55583 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:40.805531 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-62337368-0d27-40b0-a470-2b7ff0f55583 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:40.806008 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-62337368-0d27-40b0-a470-2b7ff0f55583 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:40.812954 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-97a20e3e-7a70-45f2-bfea-72e4afc95fe6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:40.816417 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-97a20e3e-7a70-45f2-bfea-72e4afc95fe6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:40.818716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-97a20e3e-7a70-45f2-bfea-72e4afc95fe6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1277 microversion: 2.20 time: 0.134932 Sep 04 01:05:40.820366 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 36/69] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:05:40 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1277 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:05:40.927416 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-933b998b-f7da-40af-b02a-a45f225e26af tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:40.928330 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-933b998b-f7da-40af-b02a-a45f225e26af tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:40.934499 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-933b998b-f7da-40af-b02a-a45f225e26af tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:40.935175 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-933b998b-f7da-40af-b02a-a45f225e26af tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:40.935768 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-933b998b-f7da-40af-b02a-a45f225e26af tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:40.936470 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-933b998b-f7da-40af-b02a-a45f225e26af tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:40.936924 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-933b998b-f7da-40af-b02a-a45f225e26af tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:40.938069 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-933b998b-f7da-40af-b02a-a45f225e26af tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:40.974518 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-62337368-0d27-40b0-a470-2b7ff0f55583 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:40.979642 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-62337368-0d27-40b0-a470-2b7ff0f55583 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1320 microversion: 2.1 time: 0.188978 Sep 04 01:05:40.980474 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 34/70] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:40 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1320 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:41.036462 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-933b998b-f7da-40af-b02a-a45f225e26af tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:41.044548 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-933b998b-f7da-40af-b02a-a45f225e26af tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1302 microversion: 2.1 time: 0.122262 Sep 04 01:05:41.045136 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 37/71] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:40 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1302 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:41.356756 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-6428dcf1-07f8-4aca-8e7b-112bed5bbd50 req-10311d60-66e0-4a0b-b2c5-62de30448931 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d296b21a-c705-4d0b-8402-19a9c6f2e3d2", "tag": "e9c83c9b-f295-4069-84cf-ff7c294ae4cd"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:05:41.367373 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-6428dcf1-07f8-4aca-8e7b-112bed5bbd50 req-10311d60-66e0-4a0b-b2c5-62de30448931 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:41.368797 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-6428dcf1-07f8-4aca-8e7b-112bed5bbd50 req-10311d60-66e0-4a0b-b2c5-62de30448931 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:41.368797 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-6428dcf1-07f8-4aca-8e7b-112bed5bbd50 req-10311d60-66e0-4a0b-b2c5-62de30448931 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:41.381653 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-6428dcf1-07f8-4aca-8e7b-112bed5bbd50 req-10311d60-66e0-4a0b-b2c5-62de30448931 service nova] Creating event network-changed:e9c83c9b-f295-4069-84cf-ff7c294ae4cd for instance d296b21a-c705-4d0b-8402-19a9c6f2e3d2 on hv-cinder-maste Sep 04 01:05:41.387561 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-6428dcf1-07f8-4aca-8e7b-112bed5bbd50 req-10311d60-66e0-4a0b-b2c5-62de30448931 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034342 Sep 04 01:05:41.388379 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 35/72] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:05:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:05:41.706722 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-1a734735-17a3-4a49-85d9-c4af29c63611 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:41.707427 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-1a734735-17a3-4a49-85d9-c4af29c63611 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:41.717342 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1a734735-17a3-4a49-85d9-c4af29c63611 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:41.721900 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1a734735-17a3-4a49-85d9-c4af29c63611 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:41.721900 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1a734735-17a3-4a49-85d9-c4af29c63611 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:41.721900 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1a734735-17a3-4a49-85d9-c4af29c63611 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:41.721900 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1a734735-17a3-4a49-85d9-c4af29c63611 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:41.722205 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1a734735-17a3-4a49-85d9-c4af29c63611 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:41.812447 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-268259e4-746d-4274-a92c-45f406b951b6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:41.813158 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-268259e4-746d-4274-a92c-45f406b951b6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:41.825837 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-268259e4-746d-4274-a92c-45f406b951b6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:41.830819 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-268259e4-746d-4274-a92c-45f406b951b6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:41.830819 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-268259e4-746d-4274-a92c-45f406b951b6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:41.830819 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-268259e4-746d-4274-a92c-45f406b951b6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:41.830819 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-268259e4-746d-4274-a92c-45f406b951b6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:41.831451 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-268259e4-746d-4274-a92c-45f406b951b6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:41.884288 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-1a734735-17a3-4a49-85d9-c4af29c63611 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:41.892785 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-1a734735-17a3-4a49-85d9-c4af29c63611 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1314 microversion: 2.1 time: 0.191938 Sep 04 01:05:41.893419 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 38/73] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:41 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1314 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:41.898392 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-f690e26e-a6bc-45b1-96d6-85b3206e444b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:41.899264 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-f690e26e-a6bc-45b1-96d6-85b3206e444b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:41.905569 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f690e26e-a6bc-45b1-96d6-85b3206e444b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:41.909407 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f690e26e-a6bc-45b1-96d6-85b3206e444b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:41.909407 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f690e26e-a6bc-45b1-96d6-85b3206e444b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:41.909407 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f690e26e-a6bc-45b1-96d6-85b3206e444b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:41.909407 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f690e26e-a6bc-45b1-96d6-85b3206e444b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:41.909761 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f690e26e-a6bc-45b1-96d6-85b3206e444b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:41.973962 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-f690e26e-a6bc-45b1-96d6-85b3206e444b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:41.982288 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-f690e26e-a6bc-45b1-96d6-85b3206e444b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:41.982465 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-268259e4-746d-4274-a92c-45f406b951b6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:41.983728 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-f690e26e-a6bc-45b1-96d6-85b3206e444b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:41.984824 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-f690e26e-a6bc-45b1-96d6-85b3206e444b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1333 microversion: 2.20 time: 0.090332 Sep 04 01:05:41.985408 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 39/74] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:05:41 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1333 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:05:41.991616 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-268259e4-746d-4274-a92c-45f406b951b6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1360 microversion: 2.1 time: 0.183501 Sep 04 01:05:41.993317 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 36/75] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:41 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1360 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:42.004461 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-394d0b4c-e445-4c89-9fb4-cdcc1fc22587 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:42.005277 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-394d0b4c-e445-4c89-9fb4-cdcc1fc22587 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:42.011223 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-394d0b4c-e445-4c89-9fb4-cdcc1fc22587 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:42.011694 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-394d0b4c-e445-4c89-9fb4-cdcc1fc22587 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:42.012124 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-394d0b4c-e445-4c89-9fb4-cdcc1fc22587 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:42.012799 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-394d0b4c-e445-4c89-9fb4-cdcc1fc22587 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:42.013302 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-394d0b4c-e445-4c89-9fb4-cdcc1fc22587 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:42.013719 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-394d0b4c-e445-4c89-9fb4-cdcc1fc22587 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:42.068565 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-3a3bdfb5-d183-45af-b954-5ec9505fe108 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:42.069554 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-3a3bdfb5-d183-45af-b954-5ec9505fe108 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:42.075985 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3a3bdfb5-d183-45af-b954-5ec9505fe108 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:42.076350 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3a3bdfb5-d183-45af-b954-5ec9505fe108 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:42.076779 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3a3bdfb5-d183-45af-b954-5ec9505fe108 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:42.077565 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3a3bdfb5-d183-45af-b954-5ec9505fe108 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:42.078008 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3a3bdfb5-d183-45af-b954-5ec9505fe108 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:42.078425 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3a3bdfb5-d183-45af-b954-5ec9505fe108 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:42.141764 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-3a3bdfb5-d183-45af-b954-5ec9505fe108 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:42.150470 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-3a3bdfb5-d183-45af-b954-5ec9505fe108 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1302 microversion: 2.1 time: 0.086551 Sep 04 01:05:42.151238 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 37/76] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:42 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1302 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:42.160848 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-394d0b4c-e445-4c89-9fb4-cdcc1fc22587 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:42.169237 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-394d0b4c-e445-4c89-9fb4-cdcc1fc22587 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1320 microversion: 2.1 time: 0.169471 Sep 04 01:05:42.169805 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 40/77] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:41 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1320 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:42.915252 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-74125af7-6d9f-4e5c-94d4-3e08d92b5cf2 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:42.915869 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-74125af7-6d9f-4e5c-94d4-3e08d92b5cf2 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:42.921745 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-74125af7-6d9f-4e5c-94d4-3e08d92b5cf2 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:42.922222 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-74125af7-6d9f-4e5c-94d4-3e08d92b5cf2 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:42.922656 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-74125af7-6d9f-4e5c-94d4-3e08d92b5cf2 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:42.923347 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-74125af7-6d9f-4e5c-94d4-3e08d92b5cf2 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:42.923792 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-74125af7-6d9f-4e5c-94d4-3e08d92b5cf2 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:42.924233 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-74125af7-6d9f-4e5c-94d4-3e08d92b5cf2 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:43.010881 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-ac777ad4-470e-4b85-9947-623bdd8b8fce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:43.011588 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-ac777ad4-470e-4b85-9947-623bdd8b8fce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:43.013585 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell0:SELECT=34 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:05:43.014503 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell0:SELECT=6 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:05:43.019854 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ac777ad4-470e-4b85-9947-623bdd8b8fce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:43.020335 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ac777ad4-470e-4b85-9947-623bdd8b8fce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:43.020768 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ac777ad4-470e-4b85-9947-623bdd8b8fce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:43.021485 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ac777ad4-470e-4b85-9947-623bdd8b8fce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:43.021944 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ac777ad4-470e-4b85-9947-623bdd8b8fce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:43.022365 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ac777ad4-470e-4b85-9947-623bdd8b8fce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:43.052797 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-74125af7-6d9f-4e5c-94d4-3e08d92b5cf2 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:43.060695 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-74125af7-6d9f-4e5c-94d4-3e08d92b5cf2 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1314 microversion: 2.1 time: 0.149283 Sep 04 01:05:43.061299 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 38/78] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:42 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1314 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:43.067210 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-152e32d1-edfe-457a-b1eb-723d4f5c1123 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:43.067849 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-152e32d1-edfe-457a-b1eb-723d4f5c1123 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:43.073999 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-152e32d1-edfe-457a-b1eb-723d4f5c1123 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:43.076850 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-152e32d1-edfe-457a-b1eb-723d4f5c1123 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:43.076850 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-152e32d1-edfe-457a-b1eb-723d4f5c1123 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:43.076850 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-152e32d1-edfe-457a-b1eb-723d4f5c1123 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:43.076850 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-152e32d1-edfe-457a-b1eb-723d4f5c1123 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:43.077159 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-152e32d1-edfe-457a-b1eb-723d4f5c1123 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:43.088307 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ac777ad4-470e-4b85-9947-623bdd8b8fce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:43.096660 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ac777ad4-470e-4b85-9947-623bdd8b8fce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:43.098296 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ac777ad4-470e-4b85-9947-623bdd8b8fce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:43.099408 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-ac777ad4-470e-4b85-9947-623bdd8b8fce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1333 microversion: 2.20 time: 0.092384 Sep 04 01:05:43.099967 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 41/79] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:05:43 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1333 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:05:43.172772 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-42384602-5e50-4ca9-be44-a7e794270707 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:43.177211 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-42384602-5e50-4ca9-be44-a7e794270707 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:43.179840 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-42384602-5e50-4ca9-be44-a7e794270707 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:43.180380 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-42384602-5e50-4ca9-be44-a7e794270707 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:43.180813 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-42384602-5e50-4ca9-be44-a7e794270707 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:43.181541 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-42384602-5e50-4ca9-be44-a7e794270707 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:43.181987 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-42384602-5e50-4ca9-be44-a7e794270707 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:43.182412 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-42384602-5e50-4ca9-be44-a7e794270707 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:43.216233 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-152e32d1-edfe-457a-b1eb-723d4f5c1123 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:43.224290 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-152e32d1-edfe-457a-b1eb-723d4f5c1123 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1360 microversion: 2.1 time: 0.160997 Sep 04 01:05:43.225098 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 39/80] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:43 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1360 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:43.230004 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-73bda2a0-b54b-4a1f-992a-7957cce8bf29 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:43.230609 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-73bda2a0-b54b-4a1f-992a-7957cce8bf29 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:43.237834 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-73bda2a0-b54b-4a1f-992a-7957cce8bf29 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:43.238639 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-73bda2a0-b54b-4a1f-992a-7957cce8bf29 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:43.239072 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-73bda2a0-b54b-4a1f-992a-7957cce8bf29 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:43.239982 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-73bda2a0-b54b-4a1f-992a-7957cce8bf29 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:43.240609 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-73bda2a0-b54b-4a1f-992a-7957cce8bf29 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:43.241289 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-73bda2a0-b54b-4a1f-992a-7957cce8bf29 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:43.249584 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-42384602-5e50-4ca9-be44-a7e794270707 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:43.258698 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-42384602-5e50-4ca9-be44-a7e794270707 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1302 microversion: 2.1 time: 0.089671 Sep 04 01:05:43.259285 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 42/81] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:43 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1302 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:05:43.368195 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-73bda2a0-b54b-4a1f-992a-7957cce8bf29 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:43.375882 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-73bda2a0-b54b-4a1f-992a-7957cce8bf29 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1320 microversion: 2.1 time: 0.149498 Sep 04 01:05:43.376441 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 40/82] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:43 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1320 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:44.084949 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-f1416f53-e6bc-4311-a473-d137a6c92492 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:44.085591 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-f1416f53-e6bc-4311-a473-d137a6c92492 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:44.092000 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f1416f53-e6bc-4311-a473-d137a6c92492 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:44.092703 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f1416f53-e6bc-4311-a473-d137a6c92492 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:44.093214 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f1416f53-e6bc-4311-a473-d137a6c92492 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:44.093916 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f1416f53-e6bc-4311-a473-d137a6c92492 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:44.094854 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f1416f53-e6bc-4311-a473-d137a6c92492 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:44.094854 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f1416f53-e6bc-4311-a473-d137a6c92492 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:44.124314 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-ac3da90b-8b66-4273-814e-3069333cfd5a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:44.124918 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-ac3da90b-8b66-4273-814e-3069333cfd5a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:44.131527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ac3da90b-8b66-4273-814e-3069333cfd5a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:44.132046 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ac3da90b-8b66-4273-814e-3069333cfd5a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:44.132607 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ac3da90b-8b66-4273-814e-3069333cfd5a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:44.133353 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ac3da90b-8b66-4273-814e-3069333cfd5a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:44.133807 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ac3da90b-8b66-4273-814e-3069333cfd5a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:44.134207 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ac3da90b-8b66-4273-814e-3069333cfd5a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:44.200124 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-ac3da90b-8b66-4273-814e-3069333cfd5a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:44.209669 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-ac3da90b-8b66-4273-814e-3069333cfd5a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:44.211440 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-ac3da90b-8b66-4273-814e-3069333cfd5a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:44.212663 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-ac3da90b-8b66-4273-814e-3069333cfd5a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1333 microversion: 2.20 time: 0.092183 Sep 04 01:05:44.213337 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 41/83] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:05:44 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1333 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:05:44.242260 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-f1416f53-e6bc-4311-a473-d137a6c92492 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:44.248894 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-516c78ad-ddf6-42a5-9172-e05bb799c6e6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:44.249529 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-516c78ad-ddf6-42a5-9172-e05bb799c6e6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:44.249984 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-f1416f53-e6bc-4311-a473-d137a6c92492 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1314 microversion: 2.1 time: 0.170119 Sep 04 01:05:44.250566 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 43/84] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:44 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1314 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:44.255646 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-516c78ad-ddf6-42a5-9172-e05bb799c6e6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:44.256120 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-516c78ad-ddf6-42a5-9172-e05bb799c6e6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:44.256553 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-516c78ad-ddf6-42a5-9172-e05bb799c6e6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:44.257266 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-516c78ad-ddf6-42a5-9172-e05bb799c6e6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:44.257717 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-516c78ad-ddf6-42a5-9172-e05bb799c6e6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:44.258307 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-516c78ad-ddf6-42a5-9172-e05bb799c6e6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:44.281388 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-7f3b4cbc-703b-4ffe-bbc1-288ed4b5dc55 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:44.282088 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-7f3b4cbc-703b-4ffe-bbc1-288ed4b5dc55 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:44.289350 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7f3b4cbc-703b-4ffe-bbc1-288ed4b5dc55 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:44.289945 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7f3b4cbc-703b-4ffe-bbc1-288ed4b5dc55 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:44.290432 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7f3b4cbc-703b-4ffe-bbc1-288ed4b5dc55 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:44.291119 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7f3b4cbc-703b-4ffe-bbc1-288ed4b5dc55 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:44.291561 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7f3b4cbc-703b-4ffe-bbc1-288ed4b5dc55 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:44.291980 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7f3b4cbc-703b-4ffe-bbc1-288ed4b5dc55 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:44.356468 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-7f3b4cbc-703b-4ffe-bbc1-288ed4b5dc55 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:44.364645 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-7f3b4cbc-703b-4ffe-bbc1-288ed4b5dc55 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1302 microversion: 2.1 time: 0.087077 Sep 04 01:05:44.365532 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 44/85] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:44 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1302 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:05:44.391876 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-516c78ad-ddf6-42a5-9172-e05bb799c6e6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:44.401035 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-1739d46d-bb5e-48f1-8d9f-358ef430b746 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:44.402166 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-1739d46d-bb5e-48f1-8d9f-358ef430b746 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:44.404024 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-516c78ad-ddf6-42a5-9172-e05bb799c6e6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1360 microversion: 2.1 time: 0.158606 Sep 04 01:05:44.411155 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1739d46d-bb5e-48f1-8d9f-358ef430b746 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:44.411846 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1739d46d-bb5e-48f1-8d9f-358ef430b746 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:44.412268 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1739d46d-bb5e-48f1-8d9f-358ef430b746 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:44.413258 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1739d46d-bb5e-48f1-8d9f-358ef430b746 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:44.413713 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1739d46d-bb5e-48f1-8d9f-358ef430b746 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:44.413928 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 42/86] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:44 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1360 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:05:44.415153 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1739d46d-bb5e-48f1-8d9f-358ef430b746 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:44.546191 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-1739d46d-bb5e-48f1-8d9f-358ef430b746 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:44.558946 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-1739d46d-bb5e-48f1-8d9f-358ef430b746 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1320 microversion: 2.1 time: 0.163306 Sep 04 01:05:44.563067 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 45/87] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:44 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1320 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:05:45.238995 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-eddf6732-d950-4af5-9864-e92a01f394fd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:45.239615 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-eddf6732-d950-4af5-9864-e92a01f394fd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:45.246489 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-eddf6732-d950-4af5-9864-e92a01f394fd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:45.246951 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-eddf6732-d950-4af5-9864-e92a01f394fd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:45.247392 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-eddf6732-d950-4af5-9864-e92a01f394fd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:45.248085 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-eddf6732-d950-4af5-9864-e92a01f394fd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:45.248570 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-eddf6732-d950-4af5-9864-e92a01f394fd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:45.248971 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-eddf6732-d950-4af5-9864-e92a01f394fd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:45.284871 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-75baf65c-a21c-49f4-b4d0-352d4d288cb7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:45.285769 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-75baf65c-a21c-49f4-b4d0-352d4d288cb7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:45.294027 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-75baf65c-a21c-49f4-b4d0-352d4d288cb7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:45.294527 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-75baf65c-a21c-49f4-b4d0-352d4d288cb7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:45.294963 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-75baf65c-a21c-49f4-b4d0-352d4d288cb7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:45.295665 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-75baf65c-a21c-49f4-b4d0-352d4d288cb7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:45.296120 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-75baf65c-a21c-49f4-b4d0-352d4d288cb7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:45.296540 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-75baf65c-a21c-49f4-b4d0-352d4d288cb7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:45.474966 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-eddf6732-d950-4af5-9864-e92a01f394fd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:45.484758 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-eddf6732-d950-4af5-9864-e92a01f394fd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:45.487255 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-eddf6732-d950-4af5-9864-e92a01f394fd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:45.488784 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-eddf6732-d950-4af5-9864-e92a01f394fd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1333 microversion: 2.20 time: 0.254647 Sep 04 01:05:45.501797 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 43/88] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:05:45 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1333 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:05:45.514176 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-5e8eb0d8-7830-4c26-831a-e02863580f37 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:45.514996 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-5e8eb0d8-7830-4c26-831a-e02863580f37 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:45.522138 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell0:SELECT=4 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:05:45.524242 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5e8eb0d8-7830-4c26-831a-e02863580f37 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:45.525116 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5e8eb0d8-7830-4c26-831a-e02863580f37 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:45.525590 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5e8eb0d8-7830-4c26-831a-e02863580f37 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:45.526313 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5e8eb0d8-7830-4c26-831a-e02863580f37 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:45.526811 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5e8eb0d8-7830-4c26-831a-e02863580f37 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:45.527232 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5e8eb0d8-7830-4c26-831a-e02863580f37 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:45.601027 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-5e8eb0d8-7830-4c26-831a-e02863580f37 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:45.613519 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-5e8eb0d8-7830-4c26-831a-e02863580f37 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1312 microversion: 2.1 time: 0.110180 Sep 04 01:05:45.625102 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 44/89] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:45 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1312 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:45.633490 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-c112e8c6-2fc8-4cc2-ae50-157cd49bb09e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:45.633490 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-c112e8c6-2fc8-4cc2-ae50-157cd49bb09e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:45.641363 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c112e8c6-2fc8-4cc2-ae50-157cd49bb09e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:45.641924 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c112e8c6-2fc8-4cc2-ae50-157cd49bb09e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:45.642363 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c112e8c6-2fc8-4cc2-ae50-157cd49bb09e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:45.643657 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c112e8c6-2fc8-4cc2-ae50-157cd49bb09e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:45.644227 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c112e8c6-2fc8-4cc2-ae50-157cd49bb09e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:45.644834 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c112e8c6-2fc8-4cc2-ae50-157cd49bb09e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:45.656282 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-75baf65c-a21c-49f4-b4d0-352d4d288cb7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:45.665098 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-75baf65c-a21c-49f4-b4d0-352d4d288cb7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1314 microversion: 2.1 time: 0.395812 Sep 04 01:05:45.665983 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 46/90] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:45 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1314 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:05:45.670219 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-26d6890d-e153-46e6-92b7-8287cbfcdad8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:45.681363 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-26d6890d-e153-46e6-92b7-8287cbfcdad8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:45.681363 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-26d6890d-e153-46e6-92b7-8287cbfcdad8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:45.681363 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-26d6890d-e153-46e6-92b7-8287cbfcdad8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:45.681363 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-26d6890d-e153-46e6-92b7-8287cbfcdad8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:45.681897 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-26d6890d-e153-46e6-92b7-8287cbfcdad8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:45.681897 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-26d6890d-e153-46e6-92b7-8287cbfcdad8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:45.681897 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-26d6890d-e153-46e6-92b7-8287cbfcdad8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:45.793646 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-c112e8c6-2fc8-4cc2-ae50-157cd49bb09e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:45.802573 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-c112e8c6-2fc8-4cc2-ae50-157cd49bb09e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1360 microversion: 2.1 time: 0.175546 Sep 04 01:05:45.804550 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 45/91] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:45 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1360 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:45.840365 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-26d6890d-e153-46e6-92b7-8287cbfcdad8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:45.853334 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-26d6890d-e153-46e6-92b7-8287cbfcdad8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1320 microversion: 2.1 time: 0.186423 Sep 04 01:05:45.854004 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 47/92] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:45 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1320 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:46.517203 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-d1c88681-0ff8-4a91-a157-d489b20a5287 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:46.519681 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-d1c88681-0ff8-4a91-a157-d489b20a5287 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:46.521764 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell0:SELECT=34 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:05:46.527403 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d1c88681-0ff8-4a91-a157-d489b20a5287 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:46.528169 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d1c88681-0ff8-4a91-a157-d489b20a5287 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:46.529275 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d1c88681-0ff8-4a91-a157-d489b20a5287 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:46.530253 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d1c88681-0ff8-4a91-a157-d489b20a5287 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:46.530982 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d1c88681-0ff8-4a91-a157-d489b20a5287 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:46.531419 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d1c88681-0ff8-4a91-a157-d489b20a5287 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:46.537756 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=100 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:05:46.641202 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-dec66ea1-b708-4de5-ae0d-ef4062ce8afa tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:46.641921 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-dec66ea1-b708-4de5-ae0d-ef4062ce8afa tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:46.648608 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dec66ea1-b708-4de5-ae0d-ef4062ce8afa tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:46.649250 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dec66ea1-b708-4de5-ae0d-ef4062ce8afa tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:46.649642 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dec66ea1-b708-4de5-ae0d-ef4062ce8afa tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:46.650446 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dec66ea1-b708-4de5-ae0d-ef4062ce8afa tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:46.650959 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dec66ea1-b708-4de5-ae0d-ef4062ce8afa tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:46.651446 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dec66ea1-b708-4de5-ae0d-ef4062ce8afa tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:46.655550 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=100 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:05:46.662329 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-d1c88681-0ff8-4a91-a157-d489b20a5287 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:46.671934 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-d1c88681-0ff8-4a91-a157-d489b20a5287 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:46.673793 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-d1c88681-0ff8-4a91-a157-d489b20a5287 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:46.675451 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-d1c88681-0ff8-4a91-a157-d489b20a5287 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1333 microversion: 2.20 time: 0.162623 Sep 04 01:05:46.676039 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 46/93] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:05:46 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1333 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:05:46.698104 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-ef96eb77-0406-4e67-a1bc-b17fa6f30ee7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:46.701582 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-ef96eb77-0406-4e67-a1bc-b17fa6f30ee7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:46.706568 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ef96eb77-0406-4e67-a1bc-b17fa6f30ee7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:46.707337 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ef96eb77-0406-4e67-a1bc-b17fa6f30ee7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:46.707985 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ef96eb77-0406-4e67-a1bc-b17fa6f30ee7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:46.708909 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ef96eb77-0406-4e67-a1bc-b17fa6f30ee7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:46.709945 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ef96eb77-0406-4e67-a1bc-b17fa6f30ee7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:46.710527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ef96eb77-0406-4e67-a1bc-b17fa6f30ee7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:46.724327 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-dec66ea1-b708-4de5-ae0d-ef4062ce8afa tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:46.736354 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-dec66ea1-b708-4de5-ae0d-ef4062ce8afa tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1300 microversion: 2.1 time: 0.097914 Sep 04 01:05:46.737378 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 48/94] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:46 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1300 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:05:46.829446 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-7e567768-cd8d-4587-b118-d7dba041db00 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:46.830071 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-7e567768-cd8d-4587-b118-d7dba041db00 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:46.837917 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7e567768-cd8d-4587-b118-d7dba041db00 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:46.838554 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7e567768-cd8d-4587-b118-d7dba041db00 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:46.839001 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7e567768-cd8d-4587-b118-d7dba041db00 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:46.839703 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7e567768-cd8d-4587-b118-d7dba041db00 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:46.840318 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7e567768-cd8d-4587-b118-d7dba041db00 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:46.840751 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7e567768-cd8d-4587-b118-d7dba041db00 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:46.916261 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-ef96eb77-0406-4e67-a1bc-b17fa6f30ee7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:46.927984 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-ef96eb77-0406-4e67-a1bc-b17fa6f30ee7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1314 microversion: 2.1 time: 0.242523 Sep 04 01:05:46.929823 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 47/95] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:46 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1314 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:05:46.937245 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-c976a915-d6ba-498c-b390-c303f2d56dcc tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:46.938092 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-c976a915-d6ba-498c-b390-c303f2d56dcc tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:46.948254 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c976a915-d6ba-498c-b390-c303f2d56dcc tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:46.948928 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c976a915-d6ba-498c-b390-c303f2d56dcc tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:46.949632 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c976a915-d6ba-498c-b390-c303f2d56dcc tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:46.950582 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c976a915-d6ba-498c-b390-c303f2d56dcc tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:46.952084 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c976a915-d6ba-498c-b390-c303f2d56dcc tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:46.952600 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c976a915-d6ba-498c-b390-c303f2d56dcc tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:47.451610 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-7e567768-cd8d-4587-b118-d7dba041db00 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:47.460799 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-7e567768-cd8d-4587-b118-d7dba041db00 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1360 microversion: 2.1 time: 0.635999 Sep 04 01:05:47.469451 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 49/96] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:46 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1360 bytes in 646 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:47.626173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-c976a915-d6ba-498c-b390-c303f2d56dcc tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:47.642054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-c976a915-d6ba-498c-b390-c303f2d56dcc tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1320 microversion: 2.1 time: 0.710840 Sep 04 01:05:47.657853 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 48/97] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:46 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1320 bytes in 713 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:47.701283 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-19c31626-a00d-48e2-b3a2-1fa56643095f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:47.702460 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-19c31626-a00d-48e2-b3a2-1fa56643095f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:47.712175 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-19c31626-a00d-48e2-b3a2-1fa56643095f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:47.714614 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-19c31626-a00d-48e2-b3a2-1fa56643095f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:47.716613 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-19c31626-a00d-48e2-b3a2-1fa56643095f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:47.719640 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-19c31626-a00d-48e2-b3a2-1fa56643095f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:47.720263 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-19c31626-a00d-48e2-b3a2-1fa56643095f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:47.720693 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-19c31626-a00d-48e2-b3a2-1fa56643095f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:47.762524 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-b62e1d39-4d6d-4877-b39e-f6e1a77601cd tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:47.763365 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-b62e1d39-4d6d-4877-b39e-f6e1a77601cd tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:47.775108 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b62e1d39-4d6d-4877-b39e-f6e1a77601cd tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:47.775790 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b62e1d39-4d6d-4877-b39e-f6e1a77601cd tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:47.776501 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b62e1d39-4d6d-4877-b39e-f6e1a77601cd tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:47.806898 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-19c31626-a00d-48e2-b3a2-1fa56643095f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:47.813653 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b62e1d39-4d6d-4877-b39e-f6e1a77601cd tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:47.815891 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b62e1d39-4d6d-4877-b39e-f6e1a77601cd tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.038s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:47.820574 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b62e1d39-4d6d-4877-b39e-f6e1a77601cd tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:47.820715 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-19c31626-a00d-48e2-b3a2-1fa56643095f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:47.825599 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-19c31626-a00d-48e2-b3a2-1fa56643095f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:47.826977 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-19c31626-a00d-48e2-b3a2-1fa56643095f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1339 microversion: 2.20 time: 0.129822 Sep 04 01:05:47.841544 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 50/98] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:05:47 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1339 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:05:47.952604 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-8238c5a4-926e-4d32-9617-eb529e232b3c tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:47.954703 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-8238c5a4-926e-4d32-9617-eb529e232b3c tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:47.962928 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8238c5a4-926e-4d32-9617-eb529e232b3c tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:47.964252 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8238c5a4-926e-4d32-9617-eb529e232b3c tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:47.964909 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8238c5a4-926e-4d32-9617-eb529e232b3c tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:47.965861 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8238c5a4-926e-4d32-9617-eb529e232b3c tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:47.966510 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8238c5a4-926e-4d32-9617-eb529e232b3c tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:47.967308 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8238c5a4-926e-4d32-9617-eb529e232b3c tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:48.063338 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-b62e1d39-4d6d-4877-b39e-f6e1a77601cd tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:48.071963 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-b62e1d39-4d6d-4877-b39e-f6e1a77601cd tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1300 microversion: 2.1 time: 0.315954 Sep 04 01:05:48.086649 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 49/99] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:47 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1300 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:48.207380 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-442d370e-b2db-45ed-83df-0e724173dfec req-c6281575-8e72-4087-a4ce-1c9cdf0f08ab service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "742cb4b8-f03c-4256-bb96-fc2cfff43734", "name": "network-vif-plugged", "status": "completed", "tag": "5d2b3e83-c7f4-4359-8491-abf0d6e3eec3"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:05:48.223858 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-442d370e-b2db-45ed-83df-0e724173dfec req-c6281575-8e72-4087-a4ce-1c9cdf0f08ab service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:48.229510 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-442d370e-b2db-45ed-83df-0e724173dfec req-c6281575-8e72-4087-a4ce-1c9cdf0f08ab service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:48.229510 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-442d370e-b2db-45ed-83df-0e724173dfec req-c6281575-8e72-4087-a4ce-1c9cdf0f08ab service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:48.246322 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-442d370e-b2db-45ed-83df-0e724173dfec req-c6281575-8e72-4087-a4ce-1c9cdf0f08ab service nova] Creating event network-vif-plugged:5d2b3e83-c7f4-4359-8491-abf0d6e3eec3 for instance 742cb4b8-f03c-4256-bb96-fc2cfff43734 on hv-cinder-maste Sep 04 01:05:48.251540 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-442d370e-b2db-45ed-83df-0e724173dfec req-c6281575-8e72-4087-a4ce-1c9cdf0f08ab service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.143417 Sep 04 01:05:48.253033 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 50/100] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:05:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:05:48.344222 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-8238c5a4-926e-4d32-9617-eb529e232b3c tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:48.352480 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-8238c5a4-926e-4d32-9617-eb529e232b3c tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1314 microversion: 2.1 time: 0.404463 Sep 04 01:05:48.353004 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 51/101] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:47 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1314 bytes in 406 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:48.495973 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-2247f6da-e730-43ae-904e-85210c89f366 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:48.496675 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-2247f6da-e730-43ae-904e-85210c89f366 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:48.504082 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2247f6da-e730-43ae-904e-85210c89f366 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:48.504765 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2247f6da-e730-43ae-904e-85210c89f366 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:48.505472 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2247f6da-e730-43ae-904e-85210c89f366 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:48.506202 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2247f6da-e730-43ae-904e-85210c89f366 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:48.506799 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2247f6da-e730-43ae-904e-85210c89f366 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:48.507236 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2247f6da-e730-43ae-904e-85210c89f366 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:48.665440 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-c7a64950-95a0-42f8-b83d-9bbdb0dbad38 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:48.666052 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-c7a64950-95a0-42f8-b83d-9bbdb0dbad38 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:48.672574 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c7a64950-95a0-42f8-b83d-9bbdb0dbad38 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:48.673060 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c7a64950-95a0-42f8-b83d-9bbdb0dbad38 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:48.673515 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c7a64950-95a0-42f8-b83d-9bbdb0dbad38 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:48.674748 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c7a64950-95a0-42f8-b83d-9bbdb0dbad38 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:48.676063 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c7a64950-95a0-42f8-b83d-9bbdb0dbad38 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:48.676063 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c7a64950-95a0-42f8-b83d-9bbdb0dbad38 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:48.835910 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-c7a64950-95a0-42f8-b83d-9bbdb0dbad38 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:48.843396 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-2247f6da-e730-43ae-904e-85210c89f366 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:48.843900 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-c7a64950-95a0-42f8-b83d-9bbdb0dbad38 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1320 microversion: 2.1 time: 0.182942 Sep 04 01:05:48.859868 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 52/102] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:48 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1320 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:48.873933 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-bbb25ebe-5eed-401e-92dc-e7aa59f135d9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:48.874157 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-2247f6da-e730-43ae-904e-85210c89f366 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1360 microversion: 2.1 time: 0.381188 Sep 04 01:05:48.874879 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 51/103] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:48 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1360 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:05:48.877861 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-bbb25ebe-5eed-401e-92dc-e7aa59f135d9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:48.884635 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bbb25ebe-5eed-401e-92dc-e7aa59f135d9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:48.885214 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bbb25ebe-5eed-401e-92dc-e7aa59f135d9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:48.886122 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bbb25ebe-5eed-401e-92dc-e7aa59f135d9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:48.887066 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bbb25ebe-5eed-401e-92dc-e7aa59f135d9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:48.887950 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bbb25ebe-5eed-401e-92dc-e7aa59f135d9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:48.888586 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bbb25ebe-5eed-401e-92dc-e7aa59f135d9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:48.987022 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-bbb25ebe-5eed-401e-92dc-e7aa59f135d9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:48.995477 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-bbb25ebe-5eed-401e-92dc-e7aa59f135d9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:48.997900 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-bbb25ebe-5eed-401e-92dc-e7aa59f135d9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:48.999237 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-bbb25ebe-5eed-401e-92dc-e7aa59f135d9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1339 microversion: 2.20 time: 0.133032 Sep 04 01:05:49.000226 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 53/104] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:05:48 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1339 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:05:49.097800 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-80a8f35c-d27c-4ddf-9e80-e99b6a704f76 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:49.098522 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-80a8f35c-d27c-4ddf-9e80-e99b6a704f76 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:49.106162 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-80a8f35c-d27c-4ddf-9e80-e99b6a704f76 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:49.106735 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-80a8f35c-d27c-4ddf-9e80-e99b6a704f76 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:49.107231 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-80a8f35c-d27c-4ddf-9e80-e99b6a704f76 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:49.107998 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-80a8f35c-d27c-4ddf-9e80-e99b6a704f76 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:49.108439 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-80a8f35c-d27c-4ddf-9e80-e99b6a704f76 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:49.108859 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-80a8f35c-d27c-4ddf-9e80-e99b6a704f76 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:49.375967 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-1b481426-053b-4fcb-80e0-ddc20ebfb86c tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:49.376587 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-1b481426-053b-4fcb-80e0-ddc20ebfb86c tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:49.383807 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1b481426-053b-4fcb-80e0-ddc20ebfb86c tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:49.384584 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1b481426-053b-4fcb-80e0-ddc20ebfb86c tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:49.385266 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1b481426-053b-4fcb-80e0-ddc20ebfb86c tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:49.386184 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1b481426-053b-4fcb-80e0-ddc20ebfb86c tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:49.387200 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1b481426-053b-4fcb-80e0-ddc20ebfb86c tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:49.387827 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1b481426-053b-4fcb-80e0-ddc20ebfb86c tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:49.550383 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-80a8f35c-d27c-4ddf-9e80-e99b6a704f76 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:49.558076 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-80a8f35c-d27c-4ddf-9e80-e99b6a704f76 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1368 microversion: 2.1 time: 0.464759 Sep 04 01:05:49.558658 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 52/105] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:49 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1368 bytes in 466 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:49.565691 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-1b481426-053b-4fcb-80e0-ddc20ebfb86c tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:49.574178 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-1b481426-053b-4fcb-80e0-ddc20ebfb86c tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1314 microversion: 2.1 time: 0.202240 Sep 04 01:05:49.574740 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 54/106] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:49 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1314 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:05:49.867023 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-d9cc56fa-8712-440d-bfed-7f2f628c045e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:49.867638 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-d9cc56fa-8712-440d-bfed-7f2f628c045e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:49.873539 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d9cc56fa-8712-440d-bfed-7f2f628c045e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:49.874018 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d9cc56fa-8712-440d-bfed-7f2f628c045e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:49.874453 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d9cc56fa-8712-440d-bfed-7f2f628c045e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:49.875137 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d9cc56fa-8712-440d-bfed-7f2f628c045e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:49.875587 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d9cc56fa-8712-440d-bfed-7f2f628c045e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:49.876009 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d9cc56fa-8712-440d-bfed-7f2f628c045e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:49.903616 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-5f472e77-08ce-4e1a-9aa0-d94fc6b4dc1f tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:49.904421 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-5f472e77-08ce-4e1a-9aa0-d94fc6b4dc1f tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:49.910778 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5f472e77-08ce-4e1a-9aa0-d94fc6b4dc1f tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:49.911253 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5f472e77-08ce-4e1a-9aa0-d94fc6b4dc1f tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:49.911682 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5f472e77-08ce-4e1a-9aa0-d94fc6b4dc1f tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:49.912362 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5f472e77-08ce-4e1a-9aa0-d94fc6b4dc1f tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:49.912799 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5f472e77-08ce-4e1a-9aa0-d94fc6b4dc1f tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:49.913239 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5f472e77-08ce-4e1a-9aa0-d94fc6b4dc1f tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:50.020157 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-d9cc56fa-8712-440d-bfed-7f2f628c045e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:50.029796 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-d9cc56fa-8712-440d-bfed-7f2f628c045e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1320 microversion: 2.1 time: 0.166255 Sep 04 01:05:50.053115 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 53/107] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:49 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1320 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:50.053115 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-82430c9b-bd30-49ff-bced-9011c20566bc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:50.053115 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-82430c9b-bd30-49ff-bced-9011c20566bc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:50.057879 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-5f472e77-08ce-4e1a-9aa0-d94fc6b4dc1f tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:50.059581 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-82430c9b-bd30-49ff-bced-9011c20566bc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:50.060371 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-82430c9b-bd30-49ff-bced-9011c20566bc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:50.061517 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-82430c9b-bd30-49ff-bced-9011c20566bc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:50.062579 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-82430c9b-bd30-49ff-bced-9011c20566bc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:50.063033 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-82430c9b-bd30-49ff-bced-9011c20566bc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:50.063497 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-82430c9b-bd30-49ff-bced-9011c20566bc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:50.067315 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-5f472e77-08ce-4e1a-9aa0-d94fc6b4dc1f tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1360 microversion: 2.1 time: 0.167391 Sep 04 01:05:50.068019 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 55/108] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:49 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1360 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:05:50.196572 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-82430c9b-bd30-49ff-bced-9011c20566bc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:50.204563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-82430c9b-bd30-49ff-bced-9011c20566bc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:50.206653 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-82430c9b-bd30-49ff-bced-9011c20566bc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:50.207693 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-82430c9b-bd30-49ff-bced-9011c20566bc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1406 microversion: 2.20 time: 0.164968 Sep 04 01:05:50.208266 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 54/109] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:05:50 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1406 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:05:50.581668 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-ccded0ff-dae9-4123-8be4-4865d0c03453 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:50.586017 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-ccded0ff-dae9-4123-8be4-4865d0c03453 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:50.588258 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ccded0ff-dae9-4123-8be4-4865d0c03453 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:50.588733 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ccded0ff-dae9-4123-8be4-4865d0c03453 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:50.589207 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ccded0ff-dae9-4123-8be4-4865d0c03453 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:50.589890 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ccded0ff-dae9-4123-8be4-4865d0c03453 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:50.590334 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ccded0ff-dae9-4123-8be4-4865d0c03453 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:50.590756 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ccded0ff-dae9-4123-8be4-4865d0c03453 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:50.597406 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-a81dc52d-4be3-46c9-9e44-0fc7f46d0457 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:50.598011 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-a81dc52d-4be3-46c9-9e44-0fc7f46d0457 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:50.603858 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a81dc52d-4be3-46c9-9e44-0fc7f46d0457 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:50.604330 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a81dc52d-4be3-46c9-9e44-0fc7f46d0457 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:50.604784 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a81dc52d-4be3-46c9-9e44-0fc7f46d0457 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:50.605517 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a81dc52d-4be3-46c9-9e44-0fc7f46d0457 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:50.605985 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a81dc52d-4be3-46c9-9e44-0fc7f46d0457 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:50.606402 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a81dc52d-4be3-46c9-9e44-0fc7f46d0457 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:50.729767 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ccded0ff-dae9-4123-8be4-4865d0c03453 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:50.738925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-ccded0ff-dae9-4123-8be4-4865d0c03453 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1368 microversion: 2.1 time: 0.161669 Sep 04 01:05:50.739808 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 56/110] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:50 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1368 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:50.744797 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-a81dc52d-4be3-46c9-9e44-0fc7f46d0457 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:50.753128 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-a81dc52d-4be3-46c9-9e44-0fc7f46d0457 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1314 microversion: 2.1 time: 0.159671 Sep 04 01:05:50.755543 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 55/111] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:50 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1314 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:05:51.057843 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-f10822e9-907c-464e-bbdd-5469f46056ad tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:51.058244 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-f10822e9-907c-464e-bbdd-5469f46056ad tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:51.066296 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f10822e9-907c-464e-bbdd-5469f46056ad tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:51.066887 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f10822e9-907c-464e-bbdd-5469f46056ad tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:51.067542 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f10822e9-907c-464e-bbdd-5469f46056ad tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:51.068456 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f10822e9-907c-464e-bbdd-5469f46056ad tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:51.069176 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f10822e9-907c-464e-bbdd-5469f46056ad tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:51.069723 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f10822e9-907c-464e-bbdd-5469f46056ad tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:51.091451 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-09acc474-5216-4e4a-8591-5e4e05c5dfbb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:51.092263 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-09acc474-5216-4e4a-8591-5e4e05c5dfbb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:51.099971 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-09acc474-5216-4e4a-8591-5e4e05c5dfbb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:51.100101 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-09acc474-5216-4e4a-8591-5e4e05c5dfbb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:51.100703 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-09acc474-5216-4e4a-8591-5e4e05c5dfbb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:51.101424 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-09acc474-5216-4e4a-8591-5e4e05c5dfbb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:51.102011 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-09acc474-5216-4e4a-8591-5e4e05c5dfbb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:51.102595 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-09acc474-5216-4e4a-8591-5e4e05c5dfbb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:51.220479 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-f10822e9-907c-464e-bbdd-5469f46056ad tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:51.234270 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-f10822e9-907c-464e-bbdd-5469f46056ad tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1320 microversion: 2.1 time: 0.180497 Sep 04 01:05:51.235077 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 57/112] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:51 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1320 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:51.246500 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-ab940a85-9219-4554-a09b-ec6931b68538 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:51.247122 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-ab940a85-9219-4554-a09b-ec6931b68538 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:51.254306 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ab940a85-9219-4554-a09b-ec6931b68538 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:51.255103 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ab940a85-9219-4554-a09b-ec6931b68538 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:51.255398 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ab940a85-9219-4554-a09b-ec6931b68538 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:51.256092 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ab940a85-9219-4554-a09b-ec6931b68538 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:51.256825 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ab940a85-9219-4554-a09b-ec6931b68538 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:51.257280 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ab940a85-9219-4554-a09b-ec6931b68538 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:51.262076 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-09acc474-5216-4e4a-8591-5e4e05c5dfbb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:51.269626 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-09acc474-5216-4e4a-8591-5e4e05c5dfbb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1360 microversion: 2.1 time: 0.181888 Sep 04 01:05:51.270547 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 56/113] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:51 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1360 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:05:51.419157 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ab940a85-9219-4554-a09b-ec6931b68538 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:51.427966 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ab940a85-9219-4554-a09b-ec6931b68538 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:51.429687 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ab940a85-9219-4554-a09b-ec6931b68538 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:51.430966 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-ab940a85-9219-4554-a09b-ec6931b68538 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1406 microversion: 2.20 time: 0.188765 Sep 04 01:05:51.432172 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 58/114] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:05:51 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1406 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:05:51.762448 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-01e6134f-3d30-473f-9228-9b6bd83ed190 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:51.763000 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-01e6134f-3d30-473f-9228-9b6bd83ed190 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:51.769792 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-01e6134f-3d30-473f-9228-9b6bd83ed190 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:51.770268 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-01e6134f-3d30-473f-9228-9b6bd83ed190 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:51.770807 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-01e6134f-3d30-473f-9228-9b6bd83ed190 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:51.771486 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-01e6134f-3d30-473f-9228-9b6bd83ed190 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:51.771934 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-01e6134f-3d30-473f-9228-9b6bd83ed190 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:51.772642 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-01e6134f-3d30-473f-9228-9b6bd83ed190 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:51.776244 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-d0cf7b27-d062-4a49-8493-1c1ea446e08e tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:51.777997 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-d0cf7b27-d062-4a49-8493-1c1ea446e08e tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:51.783899 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d0cf7b27-d062-4a49-8493-1c1ea446e08e tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:51.784635 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d0cf7b27-d062-4a49-8493-1c1ea446e08e tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:51.785067 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d0cf7b27-d062-4a49-8493-1c1ea446e08e tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:51.785882 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d0cf7b27-d062-4a49-8493-1c1ea446e08e tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:51.786402 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d0cf7b27-d062-4a49-8493-1c1ea446e08e tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:51.786823 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d0cf7b27-d062-4a49-8493-1c1ea446e08e tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:51.926801 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-01e6134f-3d30-473f-9228-9b6bd83ed190 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:51.936419 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-01e6134f-3d30-473f-9228-9b6bd83ed190 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1368 microversion: 2.1 time: 0.177845 Sep 04 01:05:51.937509 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 57/115] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:51 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1368 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:51.958190 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-d0cf7b27-d062-4a49-8493-1c1ea446e08e tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:51.966143 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-d0cf7b27-d062-4a49-8493-1c1ea446e08e tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1314 microversion: 2.1 time: 0.194306 Sep 04 01:05:51.967001 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 59/116] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:51 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1314 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:52.263450 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-a87716c4-a3bd-4e44-8b6f-dc2d86aa28a5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:52.268578 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-a87716c4-a3bd-4e44-8b6f-dc2d86aa28a5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:52.278705 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a87716c4-a3bd-4e44-8b6f-dc2d86aa28a5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:52.279497 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a87716c4-a3bd-4e44-8b6f-dc2d86aa28a5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:52.279956 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a87716c4-a3bd-4e44-8b6f-dc2d86aa28a5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:52.280680 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a87716c4-a3bd-4e44-8b6f-dc2d86aa28a5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:52.281194 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a87716c4-a3bd-4e44-8b6f-dc2d86aa28a5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:52.281782 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a87716c4-a3bd-4e44-8b6f-dc2d86aa28a5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:52.294758 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-cc7e89fb-6e92-4bed-84dc-f6ebc3f4861f tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:52.294758 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-cc7e89fb-6e92-4bed-84dc-f6ebc3f4861f tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:52.309164 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cc7e89fb-6e92-4bed-84dc-f6ebc3f4861f tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:52.309825 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cc7e89fb-6e92-4bed-84dc-f6ebc3f4861f tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:52.310269 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cc7e89fb-6e92-4bed-84dc-f6ebc3f4861f tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:52.310990 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cc7e89fb-6e92-4bed-84dc-f6ebc3f4861f tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:52.312029 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cc7e89fb-6e92-4bed-84dc-f6ebc3f4861f tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:52.312029 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cc7e89fb-6e92-4bed-84dc-f6ebc3f4861f tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:52.562358 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-cc7e89fb-6e92-4bed-84dc-f6ebc3f4861f tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:52.572357 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-cc7e89fb-6e92-4bed-84dc-f6ebc3f4861f tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1360 microversion: 2.1 time: 0.282190 Sep 04 01:05:52.573962 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 60/117] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:52 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1360 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:05:52.579765 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-17668f9a-d0a6-4867-b467-8d9057b9f3a8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:52.581207 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-17668f9a-d0a6-4867-b467-8d9057b9f3a8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:52.600152 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-17668f9a-d0a6-4867-b467-8d9057b9f3a8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:52.600660 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-17668f9a-d0a6-4867-b467-8d9057b9f3a8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:52.601461 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-17668f9a-d0a6-4867-b467-8d9057b9f3a8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:52.602219 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-17668f9a-d0a6-4867-b467-8d9057b9f3a8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:52.602706 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-17668f9a-d0a6-4867-b467-8d9057b9f3a8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:52.603126 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-17668f9a-d0a6-4867-b467-8d9057b9f3a8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:52.777028 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-17668f9a-d0a6-4867-b467-8d9057b9f3a8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:52.788094 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-17668f9a-d0a6-4867-b467-8d9057b9f3a8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:52.789972 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-17668f9a-d0a6-4867-b467-8d9057b9f3a8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:52.791526 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-17668f9a-d0a6-4867-b467-8d9057b9f3a8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1406 microversion: 2.20 time: 0.216042 Sep 04 01:05:52.792040 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 61/118] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:05:52 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1406 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:05:52.801376 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-a351f8f8-0107-4da4-8bfc-c8a677d47e12 req-54670abd-2941-4be9-af3f-54c0831623d9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2f7a7e72-1f16-4ab4-b826-d376d39be4c2", "name": "network-vif-plugged", "status": "completed", "tag": "28efc83e-9838-45a7-9fd3-3625eeff4b09"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:05:52.811757 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-a351f8f8-0107-4da4-8bfc-c8a677d47e12 req-54670abd-2941-4be9-af3f-54c0831623d9 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:52.812242 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-a351f8f8-0107-4da4-8bfc-c8a677d47e12 req-54670abd-2941-4be9-af3f-54c0831623d9 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:52.812674 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-a351f8f8-0107-4da4-8bfc-c8a677d47e12 req-54670abd-2941-4be9-af3f-54c0831623d9 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:52.825518 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-a351f8f8-0107-4da4-8bfc-c8a677d47e12 req-54670abd-2941-4be9-af3f-54c0831623d9 service nova] Creating event network-vif-plugged:28efc83e-9838-45a7-9fd3-3625eeff4b09 for instance 2f7a7e72-1f16-4ab4-b826-d376d39be4c2 on hv-cinder-maste Sep 04 01:05:52.831826 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-a351f8f8-0107-4da4-8bfc-c8a677d47e12 req-54670abd-2941-4be9-af3f-54c0831623d9 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038178 Sep 04 01:05:52.832640 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 62/119] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:05:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:05:52.877230 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-a87716c4-a3bd-4e44-8b6f-dc2d86aa28a5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:52.886624 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-a87716c4-a3bd-4e44-8b6f-dc2d86aa28a5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1320 microversion: 2.1 time: 0.627046 Sep 04 01:05:52.887794 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 58/120] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:52 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1320 bytes in 629 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:52.961343 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-e246043c-f270-42a6-aed3-3e864c4d4127 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:52.961991 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-e246043c-f270-42a6-aed3-3e864c4d4127 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:52.968217 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e246043c-f270-42a6-aed3-3e864c4d4127 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:52.968693 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e246043c-f270-42a6-aed3-3e864c4d4127 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:52.969196 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e246043c-f270-42a6-aed3-3e864c4d4127 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:52.969880 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e246043c-f270-42a6-aed3-3e864c4d4127 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:52.970320 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e246043c-f270-42a6-aed3-3e864c4d4127 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:52.971570 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e246043c-f270-42a6-aed3-3e864c4d4127 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:52.989614 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-0fa23a7d-6738-48a7-92cf-1d5ceba325bc tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:52.990487 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-0fa23a7d-6738-48a7-92cf-1d5ceba325bc tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:52.996557 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0fa23a7d-6738-48a7-92cf-1d5ceba325bc tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:52.997028 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0fa23a7d-6738-48a7-92cf-1d5ceba325bc tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:52.997477 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0fa23a7d-6738-48a7-92cf-1d5ceba325bc tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:53.008804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0fa23a7d-6738-48a7-92cf-1d5ceba325bc tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:53.008804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0fa23a7d-6738-48a7-92cf-1d5ceba325bc tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:53.008804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0fa23a7d-6738-48a7-92cf-1d5ceba325bc tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:53.141223 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e246043c-f270-42a6-aed3-3e864c4d4127 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:53.146922 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-e246043c-f270-42a6-aed3-3e864c4d4127 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1368 microversion: 2.1 time: 0.190166 Sep 04 01:05:53.147895 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 63/121] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:52 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1368 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:53.376687 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-0fa23a7d-6738-48a7-92cf-1d5ceba325bc tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:53.384704 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-0fa23a7d-6738-48a7-92cf-1d5ceba325bc tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1314 microversion: 2.1 time: 0.399163 Sep 04 01:05:53.385392 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 59/122] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:52 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1314 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:05:53.604784 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-04ef7456-133b-4f7a-9651-e97b0402e851 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:53.606101 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-04ef7456-133b-4f7a-9651-e97b0402e851 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:53.613885 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-04ef7456-133b-4f7a-9651-e97b0402e851 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:53.614373 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-04ef7456-133b-4f7a-9651-e97b0402e851 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:53.614813 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-04ef7456-133b-4f7a-9651-e97b0402e851 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:53.615536 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-04ef7456-133b-4f7a-9651-e97b0402e851 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:53.615991 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-04ef7456-133b-4f7a-9651-e97b0402e851 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:53.616419 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-04ef7456-133b-4f7a-9651-e97b0402e851 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:53.789698 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-04ef7456-133b-4f7a-9651-e97b0402e851 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:53.798222 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-04ef7456-133b-4f7a-9651-e97b0402e851 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1360 microversion: 2.1 time: 0.197167 Sep 04 01:05:53.812188 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 64/123] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:53 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1360 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:53.819554 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-87b5394a-7bd7-4903-9b32-827453ab1bba tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:53.819880 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-87b5394a-7bd7-4903-9b32-827453ab1bba tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:53.826300 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-87b5394a-7bd7-4903-9b32-827453ab1bba tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:53.826773 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-87b5394a-7bd7-4903-9b32-827453ab1bba tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:53.827222 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-87b5394a-7bd7-4903-9b32-827453ab1bba tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:53.827902 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-87b5394a-7bd7-4903-9b32-827453ab1bba tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:53.828358 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-87b5394a-7bd7-4903-9b32-827453ab1bba tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:53.828783 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-87b5394a-7bd7-4903-9b32-827453ab1bba tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:53.912341 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-e0ef16d3-9e2c-4ced-a69f-2ffaf20b0d05 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:53.913138 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-e0ef16d3-9e2c-4ced-a69f-2ffaf20b0d05 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:53.919582 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e0ef16d3-9e2c-4ced-a69f-2ffaf20b0d05 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:53.920170 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e0ef16d3-9e2c-4ced-a69f-2ffaf20b0d05 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:53.921002 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e0ef16d3-9e2c-4ced-a69f-2ffaf20b0d05 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:53.922182 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e0ef16d3-9e2c-4ced-a69f-2ffaf20b0d05 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:53.922723 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e0ef16d3-9e2c-4ced-a69f-2ffaf20b0d05 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:53.923366 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e0ef16d3-9e2c-4ced-a69f-2ffaf20b0d05 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:53.945194 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=100 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:05:54.040721 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-87b5394a-7bd7-4903-9b32-827453ab1bba tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:54.071615 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-87b5394a-7bd7-4903-9b32-827453ab1bba tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:54.073611 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-87b5394a-7bd7-4903-9b32-827453ab1bba tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:54.075213 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-87b5394a-7bd7-4903-9b32-827453ab1bba tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1406 microversion: 2.20 time: 0.262023 Sep 04 01:05:54.076384 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 60/124] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:05:53 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1406 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:05:54.082991 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-e34ed4c1-bfa6-4504-927f-3d0c01dfe9d9 req-3b72dc2b-0858-4a6b-8335-f2b18c97fb0d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "49be5a0b-f748-486e-a546-c07a8e8451a3", "tag": "fcbc60fd-1785-4f94-9436-a520ec06cdaf"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:05:54.096117 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-e34ed4c1-bfa6-4504-927f-3d0c01dfe9d9 req-3b72dc2b-0858-4a6b-8335-f2b18c97fb0d service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:54.097037 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-e34ed4c1-bfa6-4504-927f-3d0c01dfe9d9 req-3b72dc2b-0858-4a6b-8335-f2b18c97fb0d service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:54.100965 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-e34ed4c1-bfa6-4504-927f-3d0c01dfe9d9 req-3b72dc2b-0858-4a6b-8335-f2b18c97fb0d service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:54.110704 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-e34ed4c1-bfa6-4504-927f-3d0c01dfe9d9 req-3b72dc2b-0858-4a6b-8335-f2b18c97fb0d service nova] Creating event network-changed:fcbc60fd-1785-4f94-9436-a520ec06cdaf for instance 49be5a0b-f748-486e-a546-c07a8e8451a3 on hv-cinder-maste Sep 04 01:05:54.117185 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-e34ed4c1-bfa6-4504-927f-3d0c01dfe9d9 req-3b72dc2b-0858-4a6b-8335-f2b18c97fb0d service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.039272 Sep 04 01:05:54.118368 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 61/125] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:05:54 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:05:54.123052 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e0ef16d3-9e2c-4ced-a69f-2ffaf20b0d05 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:54.132541 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-e0ef16d3-9e2c-4ced-a69f-2ffaf20b0d05 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1320 microversion: 2.1 time: 0.225632 Sep 04 01:05:54.135537 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 65/126] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:53 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1320 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:05:54.174864 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-01d831c3-a92b-44f1-8321-1a26de5ef4ac tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:54.175672 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-01d831c3-a92b-44f1-8321-1a26de5ef4ac tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:54.182316 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-01d831c3-a92b-44f1-8321-1a26de5ef4ac tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:54.182795 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-01d831c3-a92b-44f1-8321-1a26de5ef4ac tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:54.187651 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-01d831c3-a92b-44f1-8321-1a26de5ef4ac tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:54.187651 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-01d831c3-a92b-44f1-8321-1a26de5ef4ac tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:54.187651 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-01d831c3-a92b-44f1-8321-1a26de5ef4ac tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:54.187651 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-01d831c3-a92b-44f1-8321-1a26de5ef4ac tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:54.335637 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-01d831c3-a92b-44f1-8321-1a26de5ef4ac tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:54.343951 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-01d831c3-a92b-44f1-8321-1a26de5ef4ac tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1368 microversion: 2.1 time: 0.173997 Sep 04 01:05:54.357839 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 62/127] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:54 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1368 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:54.407818 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-38d10a01-cfd7-4e28-9967-89c69657c3d4 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:54.413496 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-38d10a01-cfd7-4e28-9967-89c69657c3d4 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:54.415847 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-38d10a01-cfd7-4e28-9967-89c69657c3d4 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:54.416565 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-38d10a01-cfd7-4e28-9967-89c69657c3d4 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:54.417260 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-38d10a01-cfd7-4e28-9967-89c69657c3d4 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:54.418254 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-38d10a01-cfd7-4e28-9967-89c69657c3d4 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:54.418909 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-38d10a01-cfd7-4e28-9967-89c69657c3d4 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:54.419536 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-38d10a01-cfd7-4e28-9967-89c69657c3d4 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:54.592265 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-38d10a01-cfd7-4e28-9967-89c69657c3d4 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:54.608038 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-38d10a01-cfd7-4e28-9967-89c69657c3d4 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1314 microversion: 2.1 time: 0.198335 Sep 04 01:05:54.608038 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 66/128] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:54 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1314 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:54.823124 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-5c47990b-43c2-4d59-a67d-de52ef9578df tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:54.823725 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-5c47990b-43c2-4d59-a67d-de52ef9578df tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:54.829720 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5c47990b-43c2-4d59-a67d-de52ef9578df tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:54.836673 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5c47990b-43c2-4d59-a67d-de52ef9578df tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:54.836673 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5c47990b-43c2-4d59-a67d-de52ef9578df tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:54.836673 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5c47990b-43c2-4d59-a67d-de52ef9578df tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:54.836673 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5c47990b-43c2-4d59-a67d-de52ef9578df tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:54.837145 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5c47990b-43c2-4d59-a67d-de52ef9578df tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:54.847789 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-5a796e89-6707-4659-9e32-e63f32e1bdeb req-19a7a32d-5630-4553-b76a-587e500945f7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "62c4a968-ecfa-4842-8cab-8aaeb7ca7a64", "name": "network-vif-plugged", "status": "completed", "tag": "addc777a-4bdc-4153-92fa-8184c15d2cd1"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:05:54.866426 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-5a796e89-6707-4659-9e32-e63f32e1bdeb req-19a7a32d-5630-4553-b76a-587e500945f7 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:54.866426 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-5a796e89-6707-4659-9e32-e63f32e1bdeb req-19a7a32d-5630-4553-b76a-587e500945f7 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:54.866426 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-5a796e89-6707-4659-9e32-e63f32e1bdeb req-19a7a32d-5630-4553-b76a-587e500945f7 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:54.884362 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-5a796e89-6707-4659-9e32-e63f32e1bdeb req-19a7a32d-5630-4553-b76a-587e500945f7 service nova] Creating event network-vif-plugged:addc777a-4bdc-4153-92fa-8184c15d2cd1 for instance 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 on hv-cinder-maste Sep 04 01:05:54.889671 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-5a796e89-6707-4659-9e32-e63f32e1bdeb req-19a7a32d-5630-4553-b76a-587e500945f7 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046093 Sep 04 01:05:54.890479 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 67/129] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:05:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:05:54.971113 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-5c47990b-43c2-4d59-a67d-de52ef9578df tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:54.978959 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-5c47990b-43c2-4d59-a67d-de52ef9578df tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1360 microversion: 2.1 time: 0.159606 Sep 04 01:05:54.979529 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 63/130] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:54 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1360 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:55.103419 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-411f468b-b51c-48e1-b28a-a110960e7676 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:55.104104 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-411f468b-b51c-48e1-b28a-a110960e7676 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:55.110266 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-411f468b-b51c-48e1-b28a-a110960e7676 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:55.110996 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-411f468b-b51c-48e1-b28a-a110960e7676 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:55.112049 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-411f468b-b51c-48e1-b28a-a110960e7676 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:55.112729 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-411f468b-b51c-48e1-b28a-a110960e7676 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:55.113203 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-411f468b-b51c-48e1-b28a-a110960e7676 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:55.113624 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-411f468b-b51c-48e1-b28a-a110960e7676 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:55.155963 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-0074ce93-2146-4287-8d29-649c4a9e87c6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:55.156574 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-0074ce93-2146-4287-8d29-649c4a9e87c6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:55.163104 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0074ce93-2146-4287-8d29-649c4a9e87c6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:55.163583 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0074ce93-2146-4287-8d29-649c4a9e87c6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:55.164009 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0074ce93-2146-4287-8d29-649c4a9e87c6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:55.164690 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0074ce93-2146-4287-8d29-649c4a9e87c6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:55.165167 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0074ce93-2146-4287-8d29-649c4a9e87c6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:55.165630 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0074ce93-2146-4287-8d29-649c4a9e87c6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:55.168285 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=100 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:05:55.253129 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-411f468b-b51c-48e1-b28a-a110960e7676 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:55.261442 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-411f468b-b51c-48e1-b28a-a110960e7676 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:55.263382 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-411f468b-b51c-48e1-b28a-a110960e7676 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:55.264503 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-411f468b-b51c-48e1-b28a-a110960e7676 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1406 microversion: 2.20 time: 0.165361 Sep 04 01:05:55.265157 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 68/131] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:05:55 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1406 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:05:55.300711 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-0074ce93-2146-4287-8d29-649c4a9e87c6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:55.308380 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-0074ce93-2146-4287-8d29-649c4a9e87c6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1320 microversion: 2.1 time: 0.156288 Sep 04 01:05:55.308945 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 64/132] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:55 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1320 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:05:55.367708 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-52e1f863-1462-4220-945b-ba46fd7dc135 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:55.368316 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-52e1f863-1462-4220-945b-ba46fd7dc135 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:55.374325 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-52e1f863-1462-4220-945b-ba46fd7dc135 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:55.374799 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-52e1f863-1462-4220-945b-ba46fd7dc135 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:55.375227 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-52e1f863-1462-4220-945b-ba46fd7dc135 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:55.375902 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-52e1f863-1462-4220-945b-ba46fd7dc135 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:55.376346 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-52e1f863-1462-4220-945b-ba46fd7dc135 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:55.376884 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-52e1f863-1462-4220-945b-ba46fd7dc135 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:55.505146 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-52e1f863-1462-4220-945b-ba46fd7dc135 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:55.512702 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-52e1f863-1462-4220-945b-ba46fd7dc135 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1368 microversion: 2.1 time: 0.148809 Sep 04 01:05:55.518121 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 69/133] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:55 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1368 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:55.625575 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-28bdd5c9-3a4b-4130-bef8-bdb5392d3c6a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:55.626434 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-28bdd5c9-3a4b-4130-bef8-bdb5392d3c6a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:55.633052 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-28bdd5c9-3a4b-4130-bef8-bdb5392d3c6a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:55.633552 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-28bdd5c9-3a4b-4130-bef8-bdb5392d3c6a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:55.633982 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-28bdd5c9-3a4b-4130-bef8-bdb5392d3c6a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:55.634668 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-28bdd5c9-3a4b-4130-bef8-bdb5392d3c6a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:55.635115 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-28bdd5c9-3a4b-4130-bef8-bdb5392d3c6a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:55.635538 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-28bdd5c9-3a4b-4130-bef8-bdb5392d3c6a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:55.764867 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-28bdd5c9-3a4b-4130-bef8-bdb5392d3c6a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:55.773384 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-28bdd5c9-3a4b-4130-bef8-bdb5392d3c6a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1314 microversion: 2.1 time: 0.151889 Sep 04 01:05:55.773952 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 65/134] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:55 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1314 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:56.003711 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-db05582d-4339-4b5b-b51a-6129c3de2746 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:56.004321 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-db05582d-4339-4b5b-b51a-6129c3de2746 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:56.007336 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_api:SELECT=100 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:05:56.010691 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-db05582d-4339-4b5b-b51a-6129c3de2746 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:56.011171 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-db05582d-4339-4b5b-b51a-6129c3de2746 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:56.011605 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-db05582d-4339-4b5b-b51a-6129c3de2746 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:56.012288 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-db05582d-4339-4b5b-b51a-6129c3de2746 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:56.012734 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-db05582d-4339-4b5b-b51a-6129c3de2746 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:56.013187 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-db05582d-4339-4b5b-b51a-6129c3de2746 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:56.134162 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-a51fa2c7-4268-4f45-824a-3431b5331053 req-bd675b76-4814-4ddb-8af8-9a65f13b2340 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "784b2b9b-9890-4f9e-be75-2548b520e138", "tag": "e0a049e7-02dc-4e07-af75-3de7fbe36b23"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:05:56.137883 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-db05582d-4339-4b5b-b51a-6129c3de2746 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:56.143773 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-a51fa2c7-4268-4f45-824a-3431b5331053 req-bd675b76-4814-4ddb-8af8-9a65f13b2340 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:56.144240 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-a51fa2c7-4268-4f45-824a-3431b5331053 req-bd675b76-4814-4ddb-8af8-9a65f13b2340 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:56.144665 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-a51fa2c7-4268-4f45-824a-3431b5331053 req-bd675b76-4814-4ddb-8af8-9a65f13b2340 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:56.145595 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-db05582d-4339-4b5b-b51a-6129c3de2746 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1360 microversion: 2.1 time: 0.146359 Sep 04 01:05:56.146174 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 70/135] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:55 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1360 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:56.155849 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-a51fa2c7-4268-4f45-824a-3431b5331053 req-bd675b76-4814-4ddb-8af8-9a65f13b2340 service nova] Creating event network-changed:e0a049e7-02dc-4e07-af75-3de7fbe36b23 for instance 784b2b9b-9890-4f9e-be75-2548b520e138 on hv-cinder-maste Sep 04 01:05:56.161228 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-a51fa2c7-4268-4f45-824a-3431b5331053 req-bd675b76-4814-4ddb-8af8-9a65f13b2340 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030692 Sep 04 01:05:56.161814 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 66/136] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:05:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:05:56.289966 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-6b4346aa-5c57-49de-800b-23d5d490fbb9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:56.290569 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-6b4346aa-5c57-49de-800b-23d5d490fbb9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:56.296441 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6b4346aa-5c57-49de-800b-23d5d490fbb9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:56.296919 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6b4346aa-5c57-49de-800b-23d5d490fbb9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:56.297383 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6b4346aa-5c57-49de-800b-23d5d490fbb9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:56.298058 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6b4346aa-5c57-49de-800b-23d5d490fbb9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:56.299058 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6b4346aa-5c57-49de-800b-23d5d490fbb9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:56.299489 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6b4346aa-5c57-49de-800b-23d5d490fbb9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:56.331034 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-d51f5a09-1a99-4510-968c-040334f2844f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:56.331754 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-d51f5a09-1a99-4510-968c-040334f2844f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:56.338022 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d51f5a09-1a99-4510-968c-040334f2844f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:56.338503 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d51f5a09-1a99-4510-968c-040334f2844f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:56.338938 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d51f5a09-1a99-4510-968c-040334f2844f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:56.339635 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d51f5a09-1a99-4510-968c-040334f2844f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:56.340080 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d51f5a09-1a99-4510-968c-040334f2844f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:56.340502 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d51f5a09-1a99-4510-968c-040334f2844f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:56.430087 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-6b4346aa-5c57-49de-800b-23d5d490fbb9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:56.439098 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-6b4346aa-5c57-49de-800b-23d5d490fbb9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:56.440539 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-6b4346aa-5c57-49de-800b-23d5d490fbb9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:56.441673 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-6b4346aa-5c57-49de-800b-23d5d490fbb9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1406 microversion: 2.20 time: 0.155569 Sep 04 01:05:56.442220 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 71/137] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:05:56 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1406 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:05:56.477241 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-d51f5a09-1a99-4510-968c-040334f2844f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:56.480070 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-d51f5a09-1a99-4510-968c-040334f2844f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1320 microversion: 2.1 time: 0.153193 Sep 04 01:05:56.480628 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 67/138] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:56 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1320 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:05:56.539100 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-9cea94ce-393a-4a38-add9-1534e124694d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:56.539698 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-9cea94ce-393a-4a38-add9-1534e124694d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:56.545577 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9cea94ce-393a-4a38-add9-1534e124694d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:56.546051 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9cea94ce-393a-4a38-add9-1534e124694d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:56.546478 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9cea94ce-393a-4a38-add9-1534e124694d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:56.547149 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9cea94ce-393a-4a38-add9-1534e124694d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:56.547691 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9cea94ce-393a-4a38-add9-1534e124694d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:56.548126 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9cea94ce-393a-4a38-add9-1534e124694d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:56.673010 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-9cea94ce-393a-4a38-add9-1534e124694d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:56.680669 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-9cea94ce-393a-4a38-add9-1534e124694d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1368 microversion: 2.1 time: 0.145459 Sep 04 01:05:56.681262 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 72/139] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:56 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1368 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:56.794784 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-ae5983af-8fce-4b1b-89ac-2aaf79957d75 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:56.795314 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-ae5983af-8fce-4b1b-89ac-2aaf79957d75 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:56.798961 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_api:SELECT=100 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:05:56.802314 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ae5983af-8fce-4b1b-89ac-2aaf79957d75 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:56.802786 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ae5983af-8fce-4b1b-89ac-2aaf79957d75 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:56.803218 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ae5983af-8fce-4b1b-89ac-2aaf79957d75 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:56.803895 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ae5983af-8fce-4b1b-89ac-2aaf79957d75 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:56.804343 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ae5983af-8fce-4b1b-89ac-2aaf79957d75 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:56.804765 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ae5983af-8fce-4b1b-89ac-2aaf79957d75 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:56.928814 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-ae5983af-8fce-4b1b-89ac-2aaf79957d75 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:56.936305 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-ae5983af-8fce-4b1b-89ac-2aaf79957d75 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1314 microversion: 2.1 time: 0.145391 Sep 04 01:05:56.936871 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 68/140] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:56 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1314 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:57.169625 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-7f68f5e7-4975-4c55-a0e5-34bba46ccb33 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:57.170256 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-7f68f5e7-4975-4c55-a0e5-34bba46ccb33 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:57.176912 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7f68f5e7-4975-4c55-a0e5-34bba46ccb33 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:57.177412 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7f68f5e7-4975-4c55-a0e5-34bba46ccb33 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:57.177837 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7f68f5e7-4975-4c55-a0e5-34bba46ccb33 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:57.178510 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7f68f5e7-4975-4c55-a0e5-34bba46ccb33 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:57.179025 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7f68f5e7-4975-4c55-a0e5-34bba46ccb33 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:57.179479 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7f68f5e7-4975-4c55-a0e5-34bba46ccb33 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:57.308523 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-7f68f5e7-4975-4c55-a0e5-34bba46ccb33 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:57.316715 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-7f68f5e7-4975-4c55-a0e5-34bba46ccb33 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1360 microversion: 2.1 time: 0.150913 Sep 04 01:05:57.317778 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 73/141] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:57 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1360 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:57.466191 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-49353cb1-4e4a-4a48-8734-819ae4b89741 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:57.466822 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-49353cb1-4e4a-4a48-8734-819ae4b89741 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:57.473683 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-49353cb1-4e4a-4a48-8734-819ae4b89741 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:57.474154 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-49353cb1-4e4a-4a48-8734-819ae4b89741 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:57.474581 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-49353cb1-4e4a-4a48-8734-819ae4b89741 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:57.475307 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-49353cb1-4e4a-4a48-8734-819ae4b89741 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:57.475749 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-49353cb1-4e4a-4a48-8734-819ae4b89741 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:57.476166 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-49353cb1-4e4a-4a48-8734-819ae4b89741 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:57.504202 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-8bf6a356-d4f2-4863-8590-048cdfef4877 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:57.504927 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-8bf6a356-d4f2-4863-8590-048cdfef4877 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:57.511762 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8bf6a356-d4f2-4863-8590-048cdfef4877 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:57.512357 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8bf6a356-d4f2-4863-8590-048cdfef4877 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:57.512868 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8bf6a356-d4f2-4863-8590-048cdfef4877 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:57.513661 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8bf6a356-d4f2-4863-8590-048cdfef4877 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:57.514321 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8bf6a356-d4f2-4863-8590-048cdfef4877 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:57.515113 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8bf6a356-d4f2-4863-8590-048cdfef4877 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:57.684242 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-8bf6a356-d4f2-4863-8590-048cdfef4877 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:57.693089 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-8bf6a356-d4f2-4863-8590-048cdfef4877 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1320 microversion: 2.1 time: 0.193116 Sep 04 01:05:57.705031 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 74/142] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:57 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1320 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:05:57.705253 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-e11a46f0-4add-4c55-9e36-2a240cc8faa8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:57.705732 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-e11a46f0-4add-4c55-9e36-2a240cc8faa8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:57.715152 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e11a46f0-4add-4c55-9e36-2a240cc8faa8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:57.715655 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e11a46f0-4add-4c55-9e36-2a240cc8faa8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:57.716416 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e11a46f0-4add-4c55-9e36-2a240cc8faa8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:57.717346 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e11a46f0-4add-4c55-9e36-2a240cc8faa8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:57.717802 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e11a46f0-4add-4c55-9e36-2a240cc8faa8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:57.718923 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e11a46f0-4add-4c55-9e36-2a240cc8faa8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:57.917164 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e11a46f0-4add-4c55-9e36-2a240cc8faa8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:57.925544 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-e11a46f0-4add-4c55-9e36-2a240cc8faa8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1368 microversion: 2.1 time: 0.224983 Sep 04 01:05:57.926566 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 75/143] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:57 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1368 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:05:57.960209 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-4bb1e1a4-1a37-4a4f-a232-b77664d86c27 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:57.960893 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-4bb1e1a4-1a37-4a4f-a232-b77664d86c27 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:57.967450 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4bb1e1a4-1a37-4a4f-a232-b77664d86c27 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:57.968574 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4bb1e1a4-1a37-4a4f-a232-b77664d86c27 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:57.969112 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4bb1e1a4-1a37-4a4f-a232-b77664d86c27 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:57.969909 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4bb1e1a4-1a37-4a4f-a232-b77664d86c27 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:57.970428 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4bb1e1a4-1a37-4a4f-a232-b77664d86c27 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:57.970923 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4bb1e1a4-1a37-4a4f-a232-b77664d86c27 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:57.984270 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-49353cb1-4e4a-4a48-8734-819ae4b89741 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:57.993464 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-49353cb1-4e4a-4a48-8734-819ae4b89741 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:57.998039 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-49353cb1-4e4a-4a48-8734-819ae4b89741 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:57.999355 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-49353cb1-4e4a-4a48-8734-819ae4b89741 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1406 microversion: 2.20 time: 0.537221 Sep 04 01:05:57.999928 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 69/144] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:05:57 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1406 bytes in 539 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:05:58.107497 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-4bb1e1a4-1a37-4a4f-a232-b77664d86c27 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:58.116179 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-4bb1e1a4-1a37-4a4f-a232-b77664d86c27 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1314 microversion: 2.1 time: 0.160568 Sep 04 01:05:58.116887 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 76/145] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:57 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1314 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:58.343152 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-93eb6a07-3ac1-4b9e-8994-b223d4d50c8b tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:58.343754 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-93eb6a07-3ac1-4b9e-8994-b223d4d50c8b tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:58.349860 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-93eb6a07-3ac1-4b9e-8994-b223d4d50c8b tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:58.350330 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-93eb6a07-3ac1-4b9e-8994-b223d4d50c8b tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:58.350756 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-93eb6a07-3ac1-4b9e-8994-b223d4d50c8b tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:58.351433 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-93eb6a07-3ac1-4b9e-8994-b223d4d50c8b tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:58.351878 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-93eb6a07-3ac1-4b9e-8994-b223d4d50c8b tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:58.352300 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-93eb6a07-3ac1-4b9e-8994-b223d4d50c8b tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:58.488812 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-93eb6a07-3ac1-4b9e-8994-b223d4d50c8b tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:58.500654 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-93eb6a07-3ac1-4b9e-8994-b223d4d50c8b tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1360 microversion: 2.1 time: 0.161421 Sep 04 01:05:58.508680 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 70/146] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:58 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1360 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:58.707636 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-f33ee0f0-664c-48d9-97f1-54578a10b099 req-289a9905-f7a8-4b10-abd3-b45e4260c48a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d296b21a-c705-4d0b-8402-19a9c6f2e3d2", "name": "network-vif-plugged", "status": "completed", "tag": "e9c83c9b-f295-4069-84cf-ff7c294ae4cd"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:05:58.718754 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-a2381edf-aabe-4e74-aab8-851889d1f288 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:58.719730 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-a2381edf-aabe-4e74-aab8-851889d1f288 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:58.728833 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-f33ee0f0-664c-48d9-97f1-54578a10b099 req-289a9905-f7a8-4b10-abd3-b45e4260c48a service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:58.729782 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-f33ee0f0-664c-48d9-97f1-54578a10b099 req-289a9905-f7a8-4b10-abd3-b45e4260c48a service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:58.730228 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a2381edf-aabe-4e74-aab8-851889d1f288 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:58.731034 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a2381edf-aabe-4e74-aab8-851889d1f288 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:58.735266 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-f33ee0f0-664c-48d9-97f1-54578a10b099 req-289a9905-f7a8-4b10-abd3-b45e4260c48a service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:58.735618 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a2381edf-aabe-4e74-aab8-851889d1f288 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:58.735618 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a2381edf-aabe-4e74-aab8-851889d1f288 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:58.735618 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a2381edf-aabe-4e74-aab8-851889d1f288 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:58.735618 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a2381edf-aabe-4e74-aab8-851889d1f288 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:58.748529 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-f33ee0f0-664c-48d9-97f1-54578a10b099 req-289a9905-f7a8-4b10-abd3-b45e4260c48a service nova] Creating event network-vif-plugged:e9c83c9b-f295-4069-84cf-ff7c294ae4cd for instance d296b21a-c705-4d0b-8402-19a9c6f2e3d2 on hv-cinder-maste Sep 04 01:05:58.755800 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-f33ee0f0-664c-48d9-97f1-54578a10b099 req-289a9905-f7a8-4b10-abd3-b45e4260c48a service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052135 Sep 04 01:05:58.757037 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 77/147] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:05:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:05:58.874824 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-a2381edf-aabe-4e74-aab8-851889d1f288 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:58.883326 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-a2381edf-aabe-4e74-aab8-851889d1f288 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1320 microversion: 2.1 time: 0.168262 Sep 04 01:05:58.883893 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 71/148] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:58 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1320 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:58.949442 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-10217a72-971c-4d99-b5c6-8b034b608065 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:58.950177 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-10217a72-971c-4d99-b5c6-8b034b608065 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:58.956442 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-10217a72-971c-4d99-b5c6-8b034b608065 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:58.956924 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-10217a72-971c-4d99-b5c6-8b034b608065 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:58.957375 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-10217a72-971c-4d99-b5c6-8b034b608065 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:58.958064 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-10217a72-971c-4d99-b5c6-8b034b608065 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:58.958512 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-10217a72-971c-4d99-b5c6-8b034b608065 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:58.958933 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-10217a72-971c-4d99-b5c6-8b034b608065 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:59.026203 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-86ec5c6e-161c-4e62-81a7-e70826084fce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:59.026735 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-86ec5c6e-161c-4e62-81a7-e70826084fce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:59.033032 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-86ec5c6e-161c-4e62-81a7-e70826084fce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:59.033556 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-86ec5c6e-161c-4e62-81a7-e70826084fce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:59.034001 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-86ec5c6e-161c-4e62-81a7-e70826084fce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:59.035307 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-86ec5c6e-161c-4e62-81a7-e70826084fce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:59.035876 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-86ec5c6e-161c-4e62-81a7-e70826084fce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:59.036301 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-86ec5c6e-161c-4e62-81a7-e70826084fce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:59.098519 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-10217a72-971c-4d99-b5c6-8b034b608065 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:59.106295 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-10217a72-971c-4d99-b5c6-8b034b608065 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1368 microversion: 2.1 time: 0.161227 Sep 04 01:05:59.107122 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 78/149] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:58 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1368 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:59.139797 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-1e1a3a6e-879b-4059-8b94-18053df374d6 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:59.140723 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-1e1a3a6e-879b-4059-8b94-18053df374d6 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:59.147706 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1e1a3a6e-879b-4059-8b94-18053df374d6 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:59.148423 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1e1a3a6e-879b-4059-8b94-18053df374d6 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:59.148850 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1e1a3a6e-879b-4059-8b94-18053df374d6 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:59.155892 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1e1a3a6e-879b-4059-8b94-18053df374d6 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:59.156585 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1e1a3a6e-879b-4059-8b94-18053df374d6 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:59.157199 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1e1a3a6e-879b-4059-8b94-18053df374d6 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:59.252295 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-86ec5c6e-161c-4e62-81a7-e70826084fce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:59.270461 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-86ec5c6e-161c-4e62-81a7-e70826084fce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:59.274496 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-86ec5c6e-161c-4e62-81a7-e70826084fce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:59.293281 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-86ec5c6e-161c-4e62-81a7-e70826084fce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1406 microversion: 2.20 time: 0.257084 Sep 04 01:05:59.293281 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 72/150] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:05:59 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1406 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:05:59.423929 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-1e1a3a6e-879b-4059-8b94-18053df374d6 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:59.433374 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-1e1a3a6e-879b-4059-8b94-18053df374d6 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1502 microversion: 2.1 time: 0.297329 Sep 04 01:05:59.434030 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 79/151] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:59 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1502 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:05:59.528860 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-0b7becc8-1501-49e4-9103-59d2b00663e0 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:59.529670 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-0b7becc8-1501-49e4-9103-59d2b00663e0 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:59.536733 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0b7becc8-1501-49e4-9103-59d2b00663e0 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:59.537243 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0b7becc8-1501-49e4-9103-59d2b00663e0 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:59.537680 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0b7becc8-1501-49e4-9103-59d2b00663e0 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:59.538538 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0b7becc8-1501-49e4-9103-59d2b00663e0 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:59.539135 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0b7becc8-1501-49e4-9103-59d2b00663e0 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:59.539584 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0b7becc8-1501-49e4-9103-59d2b00663e0 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:59.675713 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-0b7becc8-1501-49e4-9103-59d2b00663e0 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:05:59.683329 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-0b7becc8-1501-49e4-9103-59d2b00663e0 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1360 microversion: 2.1 time: 0.159256 Sep 04 01:05:59.683896 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 73/152] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:59 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1360 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:05:59.907282 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-40637a2e-a8b0-4b32-bbff-046fc82d138d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:05:59.913215 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-40637a2e-a8b0-4b32-bbff-046fc82d138d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:05:59.915030 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-40637a2e-a8b0-4b32-bbff-046fc82d138d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:59.915527 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-40637a2e-a8b0-4b32-bbff-046fc82d138d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:59.915943 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-40637a2e-a8b0-4b32-bbff-046fc82d138d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:05:59.916627 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-40637a2e-a8b0-4b32-bbff-046fc82d138d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:05:59.917255 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-40637a2e-a8b0-4b32-bbff-046fc82d138d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:05:59.917675 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-40637a2e-a8b0-4b32-bbff-046fc82d138d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:00.132241 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-91c1f001-dbe6-44f0-bfea-e0ad6da37fca tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:00.133276 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-91c1f001-dbe6-44f0-bfea-e0ad6da37fca tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:00.140042 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-91c1f001-dbe6-44f0-bfea-e0ad6da37fca tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:00.140844 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-91c1f001-dbe6-44f0-bfea-e0ad6da37fca tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:00.142906 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-91c1f001-dbe6-44f0-bfea-e0ad6da37fca tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:00.142906 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-91c1f001-dbe6-44f0-bfea-e0ad6da37fca tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:00.142906 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-91c1f001-dbe6-44f0-bfea-e0ad6da37fca tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:00.143636 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-91c1f001-dbe6-44f0-bfea-e0ad6da37fca tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:00.250328 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-40637a2e-a8b0-4b32-bbff-046fc82d138d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:00.260996 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-40637a2e-a8b0-4b32-bbff-046fc82d138d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1320 microversion: 2.1 time: 0.357986 Sep 04 01:06:00.261633 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 80/153] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:05:59 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1320 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:00.311662 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-2320a319-6ad3-4ebd-87c5-5b7489e29a18 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:00.312269 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-2320a319-6ad3-4ebd-87c5-5b7489e29a18 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:00.319664 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2320a319-6ad3-4ebd-87c5-5b7489e29a18 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:00.320239 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2320a319-6ad3-4ebd-87c5-5b7489e29a18 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:00.320663 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2320a319-6ad3-4ebd-87c5-5b7489e29a18 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:00.321396 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2320a319-6ad3-4ebd-87c5-5b7489e29a18 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:00.322032 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2320a319-6ad3-4ebd-87c5-5b7489e29a18 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:00.322655 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2320a319-6ad3-4ebd-87c5-5b7489e29a18 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:00.357528 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-91c1f001-dbe6-44f0-bfea-e0ad6da37fca tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:00.366386 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-91c1f001-dbe6-44f0-bfea-e0ad6da37fca tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1368 microversion: 2.1 time: 0.238274 Sep 04 01:06:00.367780 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 74/154] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:00 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1368 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:06:00.464510 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-2320a319-6ad3-4ebd-87c5-5b7489e29a18 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:00.474673 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-2320a319-6ad3-4ebd-87c5-5b7489e29a18 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:00.476114 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-2320a319-6ad3-4ebd-87c5-5b7489e29a18 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:00.477245 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-2320a319-6ad3-4ebd-87c5-5b7489e29a18 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1406 microversion: 2.20 time: 0.169502 Sep 04 01:06:00.477919 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 81/155] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:06:00 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1406 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:06:00.707790 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-10cfb4e9-2000-4371-b42a-7a17ad675e50 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:00.708256 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-10cfb4e9-2000-4371-b42a-7a17ad675e50 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:00.714324 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-10cfb4e9-2000-4371-b42a-7a17ad675e50 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:00.715014 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-10cfb4e9-2000-4371-b42a-7a17ad675e50 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:00.715438 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-10cfb4e9-2000-4371-b42a-7a17ad675e50 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:00.716138 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-10cfb4e9-2000-4371-b42a-7a17ad675e50 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:00.716582 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-10cfb4e9-2000-4371-b42a-7a17ad675e50 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:00.717002 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-10cfb4e9-2000-4371-b42a-7a17ad675e50 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:00.877023 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-10cfb4e9-2000-4371-b42a-7a17ad675e50 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:00.889621 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-10cfb4e9-2000-4371-b42a-7a17ad675e50 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1360 microversion: 2.1 time: 0.182461 Sep 04 01:06:00.898038 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 75/156] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:00 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1360 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:01.287176 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-d76319dd-fbd5-4f82-bc51-88c8624011aa tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:01.291491 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-d76319dd-fbd5-4f82-bc51-88c8624011aa tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:01.293730 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d76319dd-fbd5-4f82-bc51-88c8624011aa tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:01.294208 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d76319dd-fbd5-4f82-bc51-88c8624011aa tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:01.294642 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d76319dd-fbd5-4f82-bc51-88c8624011aa tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:01.295322 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d76319dd-fbd5-4f82-bc51-88c8624011aa tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:01.295772 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d76319dd-fbd5-4f82-bc51-88c8624011aa tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:01.296191 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d76319dd-fbd5-4f82-bc51-88c8624011aa tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:01.390212 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-2fb5c86b-121d-4413-a922-459d82d9159b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:01.391087 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-2fb5c86b-121d-4413-a922-459d82d9159b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:01.398138 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2fb5c86b-121d-4413-a922-459d82d9159b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:01.398138 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2fb5c86b-121d-4413-a922-459d82d9159b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:01.398599 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2fb5c86b-121d-4413-a922-459d82d9159b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:01.399294 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2fb5c86b-121d-4413-a922-459d82d9159b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:01.399880 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2fb5c86b-121d-4413-a922-459d82d9159b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:01.400167 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2fb5c86b-121d-4413-a922-459d82d9159b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:01.466230 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-d76319dd-fbd5-4f82-bc51-88c8624011aa tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:01.474935 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-d76319dd-fbd5-4f82-bc51-88c8624011aa tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1320 microversion: 2.1 time: 0.191664 Sep 04 01:06:01.476407 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 82/157] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:01 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1320 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:01.503494 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-2fc7034d-fd31-48c7-bd15-462101f99b87 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:01.505114 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-2fc7034d-fd31-48c7-bd15-462101f99b87 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:01.512745 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2fc7034d-fd31-48c7-bd15-462101f99b87 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:01.513412 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2fc7034d-fd31-48c7-bd15-462101f99b87 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:01.513919 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2fc7034d-fd31-48c7-bd15-462101f99b87 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:01.514717 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2fc7034d-fd31-48c7-bd15-462101f99b87 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:01.515388 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2fc7034d-fd31-48c7-bd15-462101f99b87 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:01.515872 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2fc7034d-fd31-48c7-bd15-462101f99b87 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:01.537910 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=100 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:01.567083 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-2fb5c86b-121d-4413-a922-459d82d9159b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:01.574817 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-2fb5c86b-121d-4413-a922-459d82d9159b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1368 microversion: 2.1 time: 0.188704 Sep 04 01:06:01.575651 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 76/158] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:01 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1368 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:06:01.662305 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-2fc7034d-fd31-48c7-bd15-462101f99b87 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:01.670679 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-2fc7034d-fd31-48c7-bd15-462101f99b87 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:01.672175 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-2fc7034d-fd31-48c7-bd15-462101f99b87 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:01.673288 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-2fc7034d-fd31-48c7-bd15-462101f99b87 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1406 microversion: 2.20 time: 0.174282 Sep 04 01:06:01.673856 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 83/159] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:06:01 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1406 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:06:01.787801 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-f7caae5a-967d-4c22-8caa-2e851ab3305d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "47e063ee-a1bf-41b2-84b0-a8202192730f"}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:06:01.792658 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-f7caae5a-967d-4c22-8caa-2e851ab3305d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:01.800562 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f7caae5a-967d-4c22-8caa-2e851ab3305d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:01.801478 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f7caae5a-967d-4c22-8caa-2e851ab3305d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:01.802077 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f7caae5a-967d-4c22-8caa-2e851ab3305d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:01.803043 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f7caae5a-967d-4c22-8caa-2e851ab3305d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:01.803498 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f7caae5a-967d-4c22-8caa-2e851ab3305d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:01.803926 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f7caae5a-967d-4c22-8caa-2e851ab3305d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:01.912153 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b074a62e-552a-4e87-97b1-4ce477aadf5b tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:01.912766 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-b074a62e-552a-4e87-97b1-4ce477aadf5b tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:01.919061 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b074a62e-552a-4e87-97b1-4ce477aadf5b tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:01.919561 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b074a62e-552a-4e87-97b1-4ce477aadf5b tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:01.919998 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b074a62e-552a-4e87-97b1-4ce477aadf5b tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:01.920695 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b074a62e-552a-4e87-97b1-4ce477aadf5b tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:01.921173 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b074a62e-552a-4e87-97b1-4ce477aadf5b tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:01.921734 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b074a62e-552a-4e87-97b1-4ce477aadf5b tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:02.076315 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-b074a62e-552a-4e87-97b1-4ce477aadf5b tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:02.085028 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-b074a62e-552a-4e87-97b1-4ce477aadf5b tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1360 microversion: 2.1 time: 0.179072 Sep 04 01:06:02.085628 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 84/160] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:01 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1360 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:06:02.498311 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-c3363f5c-341c-44b1-80d3-7b9e6cac9b98 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:02.499111 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-c3363f5c-341c-44b1-80d3-7b9e6cac9b98 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:02.506159 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c3363f5c-341c-44b1-80d3-7b9e6cac9b98 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:02.506651 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c3363f5c-341c-44b1-80d3-7b9e6cac9b98 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:02.507122 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c3363f5c-341c-44b1-80d3-7b9e6cac9b98 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:02.507817 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c3363f5c-341c-44b1-80d3-7b9e6cac9b98 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:02.508269 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c3363f5c-341c-44b1-80d3-7b9e6cac9b98 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:02.508818 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c3363f5c-341c-44b1-80d3-7b9e6cac9b98 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:02.638647 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-c3363f5c-341c-44b1-80d3-7b9e6cac9b98 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:02.650122 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-c3363f5c-341c-44b1-80d3-7b9e6cac9b98 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1513 microversion: 2.1 time: 0.152669 Sep 04 01:06:02.650122 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 85/161] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:02 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1513 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:06:02.652460 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-e9c811a5-fd03-4e16-85d3-61c1531a764e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:02.653101 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-e9c811a5-fd03-4e16-85d3-61c1531a764e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:02.659530 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e9c811a5-fd03-4e16-85d3-61c1531a764e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:02.660094 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e9c811a5-fd03-4e16-85d3-61c1531a764e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:02.660605 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e9c811a5-fd03-4e16-85d3-61c1531a764e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:02.661338 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e9c811a5-fd03-4e16-85d3-61c1531a764e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:02.661778 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e9c811a5-fd03-4e16-85d3-61c1531a764e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:02.662265 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e9c811a5-fd03-4e16-85d3-61c1531a764e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:02.798986 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e9c811a5-fd03-4e16-85d3-61c1531a764e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:02.807184 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-e9c811a5-fd03-4e16-85d3-61c1531a764e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1368 microversion: 2.1 time: 0.158426 Sep 04 01:06:02.807748 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 86/162] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:02 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1368 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:02.812362 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-dbe02ca7-7e2e-4f31-a5e9-76d6f4aecb6e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:06:02.814610 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-dbe02ca7-7e2e-4f31-a5e9-76d6f4aecb6e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:02.820874 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dbe02ca7-7e2e-4f31-a5e9-76d6f4aecb6e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:02.821418 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dbe02ca7-7e2e-4f31-a5e9-76d6f4aecb6e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:02.821851 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dbe02ca7-7e2e-4f31-a5e9-76d6f4aecb6e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:02.824809 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dbe02ca7-7e2e-4f31-a5e9-76d6f4aecb6e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:02.824809 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dbe02ca7-7e2e-4f31-a5e9-76d6f4aecb6e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:02.824809 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dbe02ca7-7e2e-4f31-a5e9-76d6f4aecb6e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:02.888614 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.objects.instance [None req-dbe02ca7-7e2e-4f31-a5e9-76d6f4aecb6e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lazy-loading 'flavor' on Instance uuid 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 {{(pid=121036) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:06:03.208639 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-dbe02ca7-7e2e-4f31-a5e9-76d6f4aecb6e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "POST /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64/action" status: 200 len: 32 microversion: 2.1 time: 0.399612 Sep 04 01:06:03.209274 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 87/163] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Sep 4 01:06:02 2022] POST /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64/action => generated 32 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 04 01:06:03.219255 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-ff61d131-599f-4d06-a68a-5034d4560e23 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:03.219255 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-ff61d131-599f-4d06-a68a-5034d4560e23 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:03.221271 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ff61d131-599f-4d06-a68a-5034d4560e23 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:03.221744 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ff61d131-599f-4d06-a68a-5034d4560e23 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:03.222172 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ff61d131-599f-4d06-a68a-5034d4560e23 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:03.223576 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ff61d131-599f-4d06-a68a-5034d4560e23 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:03.224055 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ff61d131-599f-4d06-a68a-5034d4560e23 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:03.224472 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ff61d131-599f-4d06-a68a-5034d4560e23 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:03.387182 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ff61d131-599f-4d06-a68a-5034d4560e23 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:03.396472 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ff61d131-599f-4d06-a68a-5034d4560e23 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:03.397963 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ff61d131-599f-4d06-a68a-5034d4560e23 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:03.399078 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-ff61d131-599f-4d06-a68a-5034d4560e23 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1406 microversion: 2.20 time: 0.188673 Sep 04 01:06:03.399635 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 88/164] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:06:03 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1406 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:06:03.404032 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-8d5b18f7-c01d-4f3d-ba0d-16851c7319e4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:03.404624 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-8d5b18f7-c01d-4f3d-ba0d-16851c7319e4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:03.413876 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8d5b18f7-c01d-4f3d-ba0d-16851c7319e4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:03.416708 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8d5b18f7-c01d-4f3d-ba0d-16851c7319e4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:03.417408 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8d5b18f7-c01d-4f3d-ba0d-16851c7319e4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:03.418191 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8d5b18f7-c01d-4f3d-ba0d-16851c7319e4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:03.419772 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8d5b18f7-c01d-4f3d-ba0d-16851c7319e4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:03.420271 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8d5b18f7-c01d-4f3d-ba0d-16851c7319e4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:03.556797 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-8d5b18f7-c01d-4f3d-ba0d-16851c7319e4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:03.564534 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-8d5b18f7-c01d-4f3d-ba0d-16851c7319e4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1360 microversion: 2.1 time: 0.163899 Sep 04 01:06:03.565138 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 89/165] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:03 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1360 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:03.570611 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-16919fc9-d5ab-400b-9656-d22ddda647c7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:03.571676 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-16919fc9-d5ab-400b-9656-d22ddda647c7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:03.579168 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-16919fc9-d5ab-400b-9656-d22ddda647c7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:03.579644 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-16919fc9-d5ab-400b-9656-d22ddda647c7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:03.580071 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-16919fc9-d5ab-400b-9656-d22ddda647c7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:03.584537 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-16919fc9-d5ab-400b-9656-d22ddda647c7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:03.584537 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-16919fc9-d5ab-400b-9656-d22ddda647c7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:03.584537 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-16919fc9-d5ab-400b-9656-d22ddda647c7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:03.718840 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-16919fc9-d5ab-400b-9656-d22ddda647c7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:03.728494 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-16919fc9-d5ab-400b-9656-d22ddda647c7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.162361 Sep 04 01:06:03.729068 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 90/166] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:03 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:03.835452 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b825fa4a-30d2-4d78-b512-b5189b17bcaa tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:03.836591 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-b825fa4a-30d2-4d78-b512-b5189b17bcaa tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:03.846834 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b825fa4a-30d2-4d78-b512-b5189b17bcaa tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:03.848334 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b825fa4a-30d2-4d78-b512-b5189b17bcaa tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:03.849123 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b825fa4a-30d2-4d78-b512-b5189b17bcaa tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:03.850327 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b825fa4a-30d2-4d78-b512-b5189b17bcaa tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:03.851132 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b825fa4a-30d2-4d78-b512-b5189b17bcaa tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:03.851888 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b825fa4a-30d2-4d78-b512-b5189b17bcaa tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:04.012471 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-b825fa4a-30d2-4d78-b512-b5189b17bcaa tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:04.021343 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-b825fa4a-30d2-4d78-b512-b5189b17bcaa tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1368 microversion: 2.1 time: 0.193806 Sep 04 01:06:04.021921 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 91/167] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:03 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1368 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:04.427714 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-075b31b9-8598-4db6-a073-1beb59f080f7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:04.428230 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-075b31b9-8598-4db6-a073-1beb59f080f7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:04.434427 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-075b31b9-8598-4db6-a073-1beb59f080f7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:04.434902 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-075b31b9-8598-4db6-a073-1beb59f080f7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:04.435328 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-075b31b9-8598-4db6-a073-1beb59f080f7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:04.436017 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-075b31b9-8598-4db6-a073-1beb59f080f7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:04.436531 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-075b31b9-8598-4db6-a073-1beb59f080f7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:04.436952 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-075b31b9-8598-4db6-a073-1beb59f080f7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:04.577991 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-075b31b9-8598-4db6-a073-1beb59f080f7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:04.587739 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-075b31b9-8598-4db6-a073-1beb59f080f7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:04.589267 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-075b31b9-8598-4db6-a073-1beb59f080f7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:04.590396 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-075b31b9-8598-4db6-a073-1beb59f080f7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1406 microversion: 2.20 time: 0.167248 Sep 04 01:06:04.590981 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 92/168] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:06:04 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1406 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:06:04.595294 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-1854fff4-c36d-4d13-b6fa-098c7a31e43d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:04.595888 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-1854fff4-c36d-4d13-b6fa-098c7a31e43d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:04.601790 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1854fff4-c36d-4d13-b6fa-098c7a31e43d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:04.602266 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1854fff4-c36d-4d13-b6fa-098c7a31e43d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:04.602699 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1854fff4-c36d-4d13-b6fa-098c7a31e43d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:04.603376 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1854fff4-c36d-4d13-b6fa-098c7a31e43d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:04.603829 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1854fff4-c36d-4d13-b6fa-098c7a31e43d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:04.604247 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1854fff4-c36d-4d13-b6fa-098c7a31e43d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:04.730328 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-1854fff4-c36d-4d13-b6fa-098c7a31e43d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:04.739026 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-1854fff4-c36d-4d13-b6fa-098c7a31e43d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1360 microversion: 2.1 time: 0.147105 Sep 04 01:06:04.739602 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 93/169] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:04 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1360 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:04.755429 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-7506ff22-ce54-47cd-9ca3-339ec765a033 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:04.759254 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-7506ff22-ce54-47cd-9ca3-339ec765a033 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:04.763192 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7506ff22-ce54-47cd-9ca3-339ec765a033 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:04.763678 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7506ff22-ce54-47cd-9ca3-339ec765a033 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:04.764113 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7506ff22-ce54-47cd-9ca3-339ec765a033 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:04.764798 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7506ff22-ce54-47cd-9ca3-339ec765a033 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:04.765279 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7506ff22-ce54-47cd-9ca3-339ec765a033 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:04.765702 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7506ff22-ce54-47cd-9ca3-339ec765a033 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:04.910404 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-7506ff22-ce54-47cd-9ca3-339ec765a033 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:04.923271 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-7506ff22-ce54-47cd-9ca3-339ec765a033 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.174451 Sep 04 01:06:04.941696 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 94/170] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:04 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:06:05.043768 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-3a794643-b314-4021-81eb-e88061950f74 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:05.044402 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-3a794643-b314-4021-81eb-e88061950f74 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:05.050515 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3a794643-b314-4021-81eb-e88061950f74 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:05.051002 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3a794643-b314-4021-81eb-e88061950f74 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:05.051432 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3a794643-b314-4021-81eb-e88061950f74 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:05.052123 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3a794643-b314-4021-81eb-e88061950f74 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:05.052573 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3a794643-b314-4021-81eb-e88061950f74 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:05.052991 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3a794643-b314-4021-81eb-e88061950f74 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:05.184912 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-3a794643-b314-4021-81eb-e88061950f74 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:05.193271 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-3a794643-b314-4021-81eb-e88061950f74 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1368 microversion: 2.1 time: 0.153324 Sep 04 01:06:05.193851 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 95/171] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:05 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1368 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:06:05.616842 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-f800178d-f223-4f8e-99bd-f3568a991ecc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:05.618052 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-f800178d-f223-4f8e-99bd-f3568a991ecc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:05.625899 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f800178d-f223-4f8e-99bd-f3568a991ecc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:05.626850 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f800178d-f223-4f8e-99bd-f3568a991ecc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:05.627440 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f800178d-f223-4f8e-99bd-f3568a991ecc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:05.628463 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f800178d-f223-4f8e-99bd-f3568a991ecc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:05.628968 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f800178d-f223-4f8e-99bd-f3568a991ecc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:05.629421 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f800178d-f223-4f8e-99bd-f3568a991ecc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:05.779891 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-f800178d-f223-4f8e-99bd-f3568a991ecc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:05.787988 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-f800178d-f223-4f8e-99bd-f3568a991ecc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:05.789438 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-f800178d-f223-4f8e-99bd-f3568a991ecc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:05.790528 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-f800178d-f223-4f8e-99bd-f3568a991ecc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1406 microversion: 2.20 time: 0.179241 Sep 04 01:06:05.791078 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 96/172] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:06:05 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1406 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:06:05.801245 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-c2e9870c-c5ac-453d-afea-1069f95bbe7d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:05.802107 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-c2e9870c-c5ac-453d-afea-1069f95bbe7d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:05.808413 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c2e9870c-c5ac-453d-afea-1069f95bbe7d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:05.813404 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c2e9870c-c5ac-453d-afea-1069f95bbe7d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:05.813882 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c2e9870c-c5ac-453d-afea-1069f95bbe7d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:05.814618 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c2e9870c-c5ac-453d-afea-1069f95bbe7d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:05.815068 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c2e9870c-c5ac-453d-afea-1069f95bbe7d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:05.815844 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c2e9870c-c5ac-453d-afea-1069f95bbe7d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:05.982441 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-c2e9870c-c5ac-453d-afea-1069f95bbe7d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:05.992368 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-c2e9870c-c5ac-453d-afea-1069f95bbe7d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1360 microversion: 2.1 time: 0.195583 Sep 04 01:06:05.993303 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 97/173] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:05 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1360 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:05.998026 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-d81b774f-0cc6-4677-8241-cd76cb7bf657 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:05.998656 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-d81b774f-0cc6-4677-8241-cd76cb7bf657 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:06.004997 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d81b774f-0cc6-4677-8241-cd76cb7bf657 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:06.005698 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d81b774f-0cc6-4677-8241-cd76cb7bf657 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:06.006288 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d81b774f-0cc6-4677-8241-cd76cb7bf657 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:06.007134 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d81b774f-0cc6-4677-8241-cd76cb7bf657 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:06.007755 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d81b774f-0cc6-4677-8241-cd76cb7bf657 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:06.008171 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d81b774f-0cc6-4677-8241-cd76cb7bf657 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:06.012616 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:06.138123 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-d81b774f-0cc6-4677-8241-cd76cb7bf657 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:06.147023 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-d81b774f-0cc6-4677-8241-cd76cb7bf657 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.152746 Sep 04 01:06:06.152555 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 98/174] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:05 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:06.216872 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-ca71c120-af79-4e39-90de-beb15985bd4b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:06.217550 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-ca71c120-af79-4e39-90de-beb15985bd4b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:06.225040 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ca71c120-af79-4e39-90de-beb15985bd4b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:06.225686 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ca71c120-af79-4e39-90de-beb15985bd4b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:06.225975 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ca71c120-af79-4e39-90de-beb15985bd4b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:06.226670 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ca71c120-af79-4e39-90de-beb15985bd4b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:06.227115 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ca71c120-af79-4e39-90de-beb15985bd4b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:06.227536 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ca71c120-af79-4e39-90de-beb15985bd4b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:06.367543 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ca71c120-af79-4e39-90de-beb15985bd4b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:06.379738 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-ca71c120-af79-4e39-90de-beb15985bd4b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1368 microversion: 2.1 time: 0.166597 Sep 04 01:06:06.380303 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 99/175] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:06 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1368 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:06:06.818030 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-263e599c-4781-4781-9773-076ad4836bf5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:06.818746 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-263e599c-4781-4781-9773-076ad4836bf5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:06.829060 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-263e599c-4781-4781-9773-076ad4836bf5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:06.829598 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-263e599c-4781-4781-9773-076ad4836bf5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:06.830021 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-263e599c-4781-4781-9773-076ad4836bf5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:06.830721 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-263e599c-4781-4781-9773-076ad4836bf5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:06.831170 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-263e599c-4781-4781-9773-076ad4836bf5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:06.832195 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-263e599c-4781-4781-9773-076ad4836bf5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:06.983752 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-263e599c-4781-4781-9773-076ad4836bf5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:06.994274 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-263e599c-4781-4781-9773-076ad4836bf5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:06.995985 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-263e599c-4781-4781-9773-076ad4836bf5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:06.997313 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-263e599c-4781-4781-9773-076ad4836bf5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1406 microversion: 2.20 time: 0.184080 Sep 04 01:06:07.011406 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 100/176] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:06:06 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1406 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:06:07.016857 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-2a05f5e1-124a-4a02-951a-3596e1dce425 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:07.017823 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-2a05f5e1-124a-4a02-951a-3596e1dce425 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:07.024435 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2a05f5e1-124a-4a02-951a-3596e1dce425 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:07.024906 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2a05f5e1-124a-4a02-951a-3596e1dce425 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:07.025379 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2a05f5e1-124a-4a02-951a-3596e1dce425 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:07.026076 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2a05f5e1-124a-4a02-951a-3596e1dce425 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:07.026556 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2a05f5e1-124a-4a02-951a-3596e1dce425 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:07.026981 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2a05f5e1-124a-4a02-951a-3596e1dce425 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:07.198772 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-2a05f5e1-124a-4a02-951a-3596e1dce425 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:07.207943 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-2a05f5e1-124a-4a02-951a-3596e1dce425 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1545 microversion: 2.1 time: 0.195780 Sep 04 01:06:07.208522 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 101/177] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:07 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1545 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:06:07.213206 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-4788d110-09d8-4468-a954-0a090e31f45a req-d5561f99-607f-4cab-903d-e05300a098fe service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "784b2b9b-9890-4f9e-be75-2548b520e138", "name": "network-vif-plugged", "status": "completed", "tag": "e0a049e7-02dc-4e07-af75-3de7fbe36b23"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:06:07.222663 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-4788d110-09d8-4468-a954-0a090e31f45a req-d5561f99-607f-4cab-903d-e05300a098fe service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:07.223313 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-4788d110-09d8-4468-a954-0a090e31f45a req-d5561f99-607f-4cab-903d-e05300a098fe service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:07.223747 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-4788d110-09d8-4468-a954-0a090e31f45a req-d5561f99-607f-4cab-903d-e05300a098fe service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:07.236060 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-4788d110-09d8-4468-a954-0a090e31f45a req-d5561f99-607f-4cab-903d-e05300a098fe service nova] Creating event network-vif-plugged:e0a049e7-02dc-4e07-af75-3de7fbe36b23 for instance 784b2b9b-9890-4f9e-be75-2548b520e138 on hv-cinder-maste Sep 04 01:06:07.241795 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-4788d110-09d8-4468-a954-0a090e31f45a req-d5561f99-607f-4cab-903d-e05300a098fe service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031886 Sep 04 01:06:07.242327 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 102/178] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:06:07 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:06:07.247547 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-09a5c47b-8885-471d-bc3d-68f6748c1ac0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:07.248246 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-09a5c47b-8885-471d-bc3d-68f6748c1ac0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:07.256031 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-09a5c47b-8885-471d-bc3d-68f6748c1ac0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:07.256519 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-09a5c47b-8885-471d-bc3d-68f6748c1ac0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:07.256954 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-09a5c47b-8885-471d-bc3d-68f6748c1ac0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:07.257804 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-09a5c47b-8885-471d-bc3d-68f6748c1ac0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:07.258255 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-09a5c47b-8885-471d-bc3d-68f6748c1ac0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:07.258760 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-09a5c47b-8885-471d-bc3d-68f6748c1ac0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:07.422766 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-09a5c47b-8885-471d-bc3d-68f6748c1ac0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:07.434075 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-09a5c47b-8885-471d-bc3d-68f6748c1ac0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.190313 Sep 04 01:06:07.434891 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 103/179] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:07 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:07.439781 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-474a84b1-b635-47ea-914b-587aa8d6f26a tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:07.440465 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-474a84b1-b635-47ea-914b-587aa8d6f26a tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:07.458540 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-474a84b1-b635-47ea-914b-587aa8d6f26a tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:07.458540 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-474a84b1-b635-47ea-914b-587aa8d6f26a tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:07.458540 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-474a84b1-b635-47ea-914b-587aa8d6f26a tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:07.459310 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-474a84b1-b635-47ea-914b-587aa8d6f26a tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:07.459886 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-474a84b1-b635-47ea-914b-587aa8d6f26a tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:07.460431 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-474a84b1-b635-47ea-914b-587aa8d6f26a tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:07.563686 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-474a84b1-b635-47ea-914b-587aa8d6f26a tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.127726 Sep 04 01:06:07.564286 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 104/180] 10.222.0.42 () {58 vars in 1275 bytes} [Sun Sep 4 01:06:07 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2/os-interface => generated 284 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:06:07.569245 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-a601abc9-f71c-4fee-960f-d98d61d18d49 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:07.569857 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-a601abc9-f71c-4fee-960f-d98d61d18d49 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:07.578924 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a601abc9-f71c-4fee-960f-d98d61d18d49 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:07.579582 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a601abc9-f71c-4fee-960f-d98d61d18d49 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:07.580141 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a601abc9-f71c-4fee-960f-d98d61d18d49 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:07.580931 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a601abc9-f71c-4fee-960f-d98d61d18d49 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:07.581527 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a601abc9-f71c-4fee-960f-d98d61d18d49 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:07.582168 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a601abc9-f71c-4fee-960f-d98d61d18d49 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:08.072402 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-a601abc9-f71c-4fee-960f-d98d61d18d49 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:08.088273 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-a601abc9-f71c-4fee-960f-d98d61d18d49 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1368 microversion: 2.1 time: 0.522545 Sep 04 01:06:08.088937 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 105/181] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:07 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1368 bytes in 524 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:08.106388 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-6decfef6-0547-4313-8eca-7861c9c0af28 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:08.107434 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-6decfef6-0547-4313-8eca-7861c9c0af28 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:08.115264 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6decfef6-0547-4313-8eca-7861c9c0af28 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:08.117416 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6decfef6-0547-4313-8eca-7861c9c0af28 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:08.117416 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6decfef6-0547-4313-8eca-7861c9c0af28 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:08.117416 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6decfef6-0547-4313-8eca-7861c9c0af28 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:08.117849 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6decfef6-0547-4313-8eca-7861c9c0af28 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:08.118281 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6decfef6-0547-4313-8eca-7861c9c0af28 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:08.598412 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-f7caae5a-967d-4c22-8caa-2e851ab3305d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "POST /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 6.813258 Sep 04 01:06:08.616687 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 77/182] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Sep 4 01:06:01 2022] POST /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734/os-volume_attachments => generated 194 bytes in 6834 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:06:08.631414 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-99fa0815-e0c4-4e49-88db-c2235b417f3b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:08.637738 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-99fa0815-e0c4-4e49-88db-c2235b417f3b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:08.652411 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-99fa0815-e0c4-4e49-88db-c2235b417f3b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:08.653020 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-99fa0815-e0c4-4e49-88db-c2235b417f3b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:08.653702 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-99fa0815-e0c4-4e49-88db-c2235b417f3b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:08.654659 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-99fa0815-e0c4-4e49-88db-c2235b417f3b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:08.655469 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-99fa0815-e0c4-4e49-88db-c2235b417f3b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:08.656112 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-99fa0815-e0c4-4e49-88db-c2235b417f3b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:08.823466 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-6decfef6-0547-4313-8eca-7861c9c0af28 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:08.837938 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-6decfef6-0547-4313-8eca-7861c9c0af28 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:08.840899 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-6decfef6-0547-4313-8eca-7861c9c0af28 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:08.843297 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-6decfef6-0547-4313-8eca-7861c9c0af28 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1406 microversion: 2.20 time: 0.748748 Sep 04 01:06:08.845395 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 106/183] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:06:08 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1406 bytes in 756 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:06:08.943723 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-99fa0815-e0c4-4e49-88db-c2235b417f3b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:08.953383 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-99fa0815-e0c4-4e49-88db-c2235b417f3b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.332711 Sep 04 01:06:08.954001 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 78/184] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:08 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:09.113815 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-047b6119-d276-45c2-b029-fc5b0c74a704 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:09.114461 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-047b6119-d276-45c2-b029-fc5b0c74a704 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:09.121870 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-047b6119-d276-45c2-b029-fc5b0c74a704 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:09.123034 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-047b6119-d276-45c2-b029-fc5b0c74a704 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:09.123632 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-047b6119-d276-45c2-b029-fc5b0c74a704 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:09.124530 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-047b6119-d276-45c2-b029-fc5b0c74a704 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:09.125930 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-047b6119-d276-45c2-b029-fc5b0c74a704 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:09.126532 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-047b6119-d276-45c2-b029-fc5b0c74a704 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:09.368773 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-047b6119-d276-45c2-b029-fc5b0c74a704 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:09.377603 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-047b6119-d276-45c2-b029-fc5b0c74a704 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1368 microversion: 2.1 time: 0.270030 Sep 04 01:06:09.379135 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 107/185] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:09 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1368 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:09.632567 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-c6c0af38-825f-4145-b5a8-1c5eb9ddac90 req-4c7e5101-d99c-4981-9faf-9ac77a28be74 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d296b21a-c705-4d0b-8402-19a9c6f2e3d2", "tag": "e9c83c9b-f295-4069-84cf-ff7c294ae4cd"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:06:09.646360 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-c6c0af38-825f-4145-b5a8-1c5eb9ddac90 req-4c7e5101-d99c-4981-9faf-9ac77a28be74 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:09.647069 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-c6c0af38-825f-4145-b5a8-1c5eb9ddac90 req-4c7e5101-d99c-4981-9faf-9ac77a28be74 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:09.647510 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-c6c0af38-825f-4145-b5a8-1c5eb9ddac90 req-4c7e5101-d99c-4981-9faf-9ac77a28be74 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:09.665197 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-c6c0af38-825f-4145-b5a8-1c5eb9ddac90 req-4c7e5101-d99c-4981-9faf-9ac77a28be74 service nova] Creating event network-changed:e9c83c9b-f295-4069-84cf-ff7c294ae4cd for instance d296b21a-c705-4d0b-8402-19a9c6f2e3d2 on hv-cinder-maste Sep 04 01:06:09.670469 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-c6c0af38-825f-4145-b5a8-1c5eb9ddac90 req-4c7e5101-d99c-4981-9faf-9ac77a28be74 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042955 Sep 04 01:06:09.671091 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 79/186] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:06:09 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:06:09.880626 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-ad7ca697-7ef2-49ff-a2c6-05ac8d0689d7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:09.881674 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-ad7ca697-7ef2-49ff-a2c6-05ac8d0689d7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:09.894625 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ad7ca697-7ef2-49ff-a2c6-05ac8d0689d7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:09.896644 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ad7ca697-7ef2-49ff-a2c6-05ac8d0689d7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:09.901501 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ad7ca697-7ef2-49ff-a2c6-05ac8d0689d7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:09.905884 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ad7ca697-7ef2-49ff-a2c6-05ac8d0689d7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:09.905884 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ad7ca697-7ef2-49ff-a2c6-05ac8d0689d7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:09.905884 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ad7ca697-7ef2-49ff-a2c6-05ac8d0689d7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:09.976873 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-1cba1e03-70a2-4f16-b891-88d7470d13f3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:09.977864 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-1cba1e03-70a2-4f16-b891-88d7470d13f3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:09.985123 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1cba1e03-70a2-4f16-b891-88d7470d13f3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:09.986104 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1cba1e03-70a2-4f16-b891-88d7470d13f3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:09.986755 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1cba1e03-70a2-4f16-b891-88d7470d13f3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:09.987671 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1cba1e03-70a2-4f16-b891-88d7470d13f3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:09.988287 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1cba1e03-70a2-4f16-b891-88d7470d13f3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:09.988758 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1cba1e03-70a2-4f16-b891-88d7470d13f3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:10.061643 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ad7ca697-7ef2-49ff-a2c6-05ac8d0689d7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:10.073100 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ad7ca697-7ef2-49ff-a2c6-05ac8d0689d7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:10.074790 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ad7ca697-7ef2-49ff-a2c6-05ac8d0689d7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:10.089165 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-ad7ca697-7ef2-49ff-a2c6-05ac8d0689d7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1406 microversion: 2.20 time: 0.203271 Sep 04 01:06:10.090072 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 108/187] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:06:09 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1406 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:06:10.254724 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-1cba1e03-70a2-4f16-b891-88d7470d13f3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:10.264298 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:10.267334 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-1cba1e03-70a2-4f16-b891-88d7470d13f3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.294812 Sep 04 01:06:10.268101 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 80/188] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:09 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:06:10.405036 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-62e90528-e4da-43d5-a8dd-2b496a65cb02 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:10.405796 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-62e90528-e4da-43d5-a8dd-2b496a65cb02 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:10.417510 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-62e90528-e4da-43d5-a8dd-2b496a65cb02 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:10.418518 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-62e90528-e4da-43d5-a8dd-2b496a65cb02 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:10.420467 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-62e90528-e4da-43d5-a8dd-2b496a65cb02 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:10.421408 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-62e90528-e4da-43d5-a8dd-2b496a65cb02 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:10.424054 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-62e90528-e4da-43d5-a8dd-2b496a65cb02 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:10.424661 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-62e90528-e4da-43d5-a8dd-2b496a65cb02 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:10.617532 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-62e90528-e4da-43d5-a8dd-2b496a65cb02 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:10.627396 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-62e90528-e4da-43d5-a8dd-2b496a65cb02 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1368 microversion: 2.1 time: 0.226397 Sep 04 01:06:10.629744 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 109/189] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:10 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1368 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:11.103383 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-660eb363-7deb-4961-a3f3-fffe8ae7fc85 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:11.104012 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-660eb363-7deb-4961-a3f3-fffe8ae7fc85 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:11.113654 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-660eb363-7deb-4961-a3f3-fffe8ae7fc85 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:11.114400 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-660eb363-7deb-4961-a3f3-fffe8ae7fc85 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:11.119693 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-660eb363-7deb-4961-a3f3-fffe8ae7fc85 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:11.127704 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-660eb363-7deb-4961-a3f3-fffe8ae7fc85 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:11.132126 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-660eb363-7deb-4961-a3f3-fffe8ae7fc85 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:11.134501 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-660eb363-7deb-4961-a3f3-fffe8ae7fc85 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:11.292793 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-183ca6ab-24b4-487d-a160-7e706b21bff4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:11.293526 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-183ca6ab-24b4-487d-a160-7e706b21bff4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:11.304066 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-183ca6ab-24b4-487d-a160-7e706b21bff4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:11.304995 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-183ca6ab-24b4-487d-a160-7e706b21bff4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:11.309314 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-183ca6ab-24b4-487d-a160-7e706b21bff4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:11.309314 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-183ca6ab-24b4-487d-a160-7e706b21bff4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:11.309314 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-183ca6ab-24b4-487d-a160-7e706b21bff4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:11.309314 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-183ca6ab-24b4-487d-a160-7e706b21bff4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:11.537425 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-660eb363-7deb-4961-a3f3-fffe8ae7fc85 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:11.558068 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-660eb363-7deb-4961-a3f3-fffe8ae7fc85 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:11.558068 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-660eb363-7deb-4961-a3f3-fffe8ae7fc85 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:11.568524 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-660eb363-7deb-4961-a3f3-fffe8ae7fc85 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1406 microversion: 2.20 time: 0.468863 Sep 04 01:06:11.572360 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 81/190] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:06:11 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1406 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:06:11.603942 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-183ca6ab-24b4-487d-a160-7e706b21bff4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:11.615057 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-183ca6ab-24b4-487d-a160-7e706b21bff4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.323072 Sep 04 01:06:11.615688 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 110/191] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:11 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:06:11.656225 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-6ed4c137-4726-4bb1-9a31-1d973e0adfb8 req-9e3f3677-35c7-4fe4-af10-46709bbc97b1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "49be5a0b-f748-486e-a546-c07a8e8451a3", "name": "network-vif-plugged", "status": "completed", "tag": "fcbc60fd-1785-4f94-9436-a520ec06cdaf"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:06:11.657853 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-fda6ce06-bd36-4cd8-adb5-72023aa605dc tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:11.658473 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-fda6ce06-bd36-4cd8-adb5-72023aa605dc tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:11.678206 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-6ed4c137-4726-4bb1-9a31-1d973e0adfb8 req-9e3f3677-35c7-4fe4-af10-46709bbc97b1 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:11.680065 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-6ed4c137-4726-4bb1-9a31-1d973e0adfb8 req-9e3f3677-35c7-4fe4-af10-46709bbc97b1 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:11.681201 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-6ed4c137-4726-4bb1-9a31-1d973e0adfb8 req-9e3f3677-35c7-4fe4-af10-46709bbc97b1 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:11.682826 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fda6ce06-bd36-4cd8-adb5-72023aa605dc tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:11.683339 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fda6ce06-bd36-4cd8-adb5-72023aa605dc tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:11.683787 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fda6ce06-bd36-4cd8-adb5-72023aa605dc tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:11.684514 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fda6ce06-bd36-4cd8-adb5-72023aa605dc tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:11.684974 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fda6ce06-bd36-4cd8-adb5-72023aa605dc tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:11.685432 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fda6ce06-bd36-4cd8-adb5-72023aa605dc tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:11.700747 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-6ed4c137-4726-4bb1-9a31-1d973e0adfb8 req-9e3f3677-35c7-4fe4-af10-46709bbc97b1 service nova] Creating event network-vif-plugged:fcbc60fd-1785-4f94-9436-a520ec06cdaf for instance 49be5a0b-f748-486e-a546-c07a8e8451a3 on hv-cinder-maste Sep 04 01:06:11.707404 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-6ed4c137-4726-4bb1-9a31-1d973e0adfb8 req-9e3f3677-35c7-4fe4-af10-46709bbc97b1 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.053960 Sep 04 01:06:11.708392 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 111/192] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:06:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:06:12.063823 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-fda6ce06-bd36-4cd8-adb5-72023aa605dc tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:12.078869 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-fda6ce06-bd36-4cd8-adb5-72023aa605dc tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1559 microversion: 2.1 time: 0.425052 Sep 04 01:06:12.079921 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 82/193] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:11 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1559 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:12.119108 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-e57e1184-ae54-4759-95b6-05330e6da3f4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:12.119108 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-e57e1184-ae54-4759-95b6-05330e6da3f4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:12.124875 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e57e1184-ae54-4759-95b6-05330e6da3f4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:12.125453 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e57e1184-ae54-4759-95b6-05330e6da3f4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:12.125974 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e57e1184-ae54-4759-95b6-05330e6da3f4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:12.127164 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e57e1184-ae54-4759-95b6-05330e6da3f4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:12.128375 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e57e1184-ae54-4759-95b6-05330e6da3f4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:12.128851 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e57e1184-ae54-4759-95b6-05330e6da3f4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:12.282393 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-e57e1184-ae54-4759-95b6-05330e6da3f4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.176226 Sep 04 01:06:12.283561 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 112/194] 10.222.0.42 () {58 vars in 1275 bytes} [Sun Sep 4 01:06:12 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138/os-interface => generated 285 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:06:12.594702 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-1c6ebd4e-d8b5-446c-860c-59c9825b7b20 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:12.595185 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-1c6ebd4e-d8b5-446c-860c-59c9825b7b20 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:12.606851 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1c6ebd4e-d8b5-446c-860c-59c9825b7b20 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:12.610982 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1c6ebd4e-d8b5-446c-860c-59c9825b7b20 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:12.610982 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1c6ebd4e-d8b5-446c-860c-59c9825b7b20 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:12.610982 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1c6ebd4e-d8b5-446c-860c-59c9825b7b20 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:12.610982 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1c6ebd4e-d8b5-446c-860c-59c9825b7b20 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:12.611301 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1c6ebd4e-d8b5-446c-860c-59c9825b7b20 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:12.646479 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-c064b560-c5c1-4ea5-98e0-f761c5e6a23f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:12.648532 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-c064b560-c5c1-4ea5-98e0-f761c5e6a23f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:12.662988 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c064b560-c5c1-4ea5-98e0-f761c5e6a23f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:12.663885 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c064b560-c5c1-4ea5-98e0-f761c5e6a23f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:12.664495 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c064b560-c5c1-4ea5-98e0-f761c5e6a23f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:12.666978 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c064b560-c5c1-4ea5-98e0-f761c5e6a23f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:12.667551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c064b560-c5c1-4ea5-98e0-f761c5e6a23f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:12.669632 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c064b560-c5c1-4ea5-98e0-f761c5e6a23f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:12.836594 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-c064b560-c5c1-4ea5-98e0-f761c5e6a23f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:12.845106 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-c064b560-c5c1-4ea5-98e0-f761c5e6a23f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.203242 Sep 04 01:06:12.846335 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 113/195] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:12 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:13.148962 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-1c6ebd4e-d8b5-446c-860c-59c9825b7b20 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:13.160458 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-1c6ebd4e-d8b5-446c-860c-59c9825b7b20 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:13.162356 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-1c6ebd4e-d8b5-446c-860c-59c9825b7b20 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:13.164365 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-1c6ebd4e-d8b5-446c-860c-59c9825b7b20 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1406 microversion: 2.20 time: 0.573766 Sep 04 01:06:13.166607 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 83/196] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:06:12 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1406 bytes in 576 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:06:13.869601 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-d0c2b388-4a48-4dcb-896d-05f584229b31 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:13.870209 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-d0c2b388-4a48-4dcb-896d-05f584229b31 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:13.876518 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d0c2b388-4a48-4dcb-896d-05f584229b31 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:13.877122 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d0c2b388-4a48-4dcb-896d-05f584229b31 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:13.877645 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d0c2b388-4a48-4dcb-896d-05f584229b31 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:13.878673 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d0c2b388-4a48-4dcb-896d-05f584229b31 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:13.879177 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d0c2b388-4a48-4dcb-896d-05f584229b31 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:13.879733 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d0c2b388-4a48-4dcb-896d-05f584229b31 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:14.036109 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-d0c2b388-4a48-4dcb-896d-05f584229b31 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:14.046858 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-d0c2b388-4a48-4dcb-896d-05f584229b31 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.180669 Sep 04 01:06:14.047465 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 114/197] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:13 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:14.191453 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-a451beac-1930-4023-b2b7-1cc75f4ec7d4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:14.192162 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-a451beac-1930-4023-b2b7-1cc75f4ec7d4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:14.198463 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a451beac-1930-4023-b2b7-1cc75f4ec7d4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:14.199022 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a451beac-1930-4023-b2b7-1cc75f4ec7d4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:14.199551 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a451beac-1930-4023-b2b7-1cc75f4ec7d4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:14.200452 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a451beac-1930-4023-b2b7-1cc75f4ec7d4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:14.201172 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a451beac-1930-4023-b2b7-1cc75f4ec7d4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:14.201537 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a451beac-1930-4023-b2b7-1cc75f4ec7d4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:14.343108 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-cb57f9ec-80ee-4777-840d-81479a980495 req-ece83de0-0b0d-4048-8bdc-9c9b282bfd28 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "784b2b9b-9890-4f9e-be75-2548b520e138", "tag": "e0a049e7-02dc-4e07-af75-3de7fbe36b23"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:06:14.365474 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-cb57f9ec-80ee-4777-840d-81479a980495 req-ece83de0-0b0d-4048-8bdc-9c9b282bfd28 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:14.365474 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-cb57f9ec-80ee-4777-840d-81479a980495 req-ece83de0-0b0d-4048-8bdc-9c9b282bfd28 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:14.365474 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-cb57f9ec-80ee-4777-840d-81479a980495 req-ece83de0-0b0d-4048-8bdc-9c9b282bfd28 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:14.388008 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-cb57f9ec-80ee-4777-840d-81479a980495 req-ece83de0-0b0d-4048-8bdc-9c9b282bfd28 service nova] Creating event network-changed:e0a049e7-02dc-4e07-af75-3de7fbe36b23 for instance 784b2b9b-9890-4f9e-be75-2548b520e138 on hv-cinder-maste Sep 04 01:06:14.412330 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-cb57f9ec-80ee-4777-840d-81479a980495 req-ece83de0-0b0d-4048-8bdc-9c9b282bfd28 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.072831 Sep 04 01:06:14.414639 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 115/198] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:06:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:06:14.431791 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-a451beac-1930-4023-b2b7-1cc75f4ec7d4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:14.441225 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-a451beac-1930-4023-b2b7-1cc75f4ec7d4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:14.442866 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-a451beac-1930-4023-b2b7-1cc75f4ec7d4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:14.444000 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-a451beac-1930-4023-b2b7-1cc75f4ec7d4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1406 microversion: 2.20 time: 0.256691 Sep 04 01:06:14.445129 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 84/199] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:06:14 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1406 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:06:14.582592 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-cc8296d1-a709-4d01-8a46-d19fb1487537 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-1967114265", "imageRef": "a11315c3-ce81-478b-91e4-5353f189dfa7", "flavorRef": "452"}} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:06:14.634565 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.network.neutron [None req-cc8296d1-a709-4d01-8a46-d19fb1487537 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] validate_networks() for None {{(pid=121036) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 04 01:06:14.988140 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-cc8296d1-a709-4d01-8a46-d19fb1487537 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Flavor limits 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 04 01:06:14.988704 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-cc8296d1-a709-4d01-8a46-d19fb1487537 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Image limits 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 04 01:06:14.989705 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-cc8296d1-a709-4d01-8a46-d19fb1487537 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Flavor pref 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 04 01:06:14.990443 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-cc8296d1-a709-4d01-8a46-d19fb1487537 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Image pref 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 04 01:06:14.990855 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-cc8296d1-a709-4d01-8a46-d19fb1487537 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 04 01:06:15.003427 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cc8296d1-a709-4d01-8a46-d19fb1487537 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:15.004198 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cc8296d1-a709-4d01-8a46-d19fb1487537 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:15.004779 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cc8296d1-a709-4d01-8a46-d19fb1487537 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:15.006154 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cc8296d1-a709-4d01-8a46-d19fb1487537 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:15.006768 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cc8296d1-a709-4d01-8a46-d19fb1487537 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:15.007347 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cc8296d1-a709-4d01-8a46-d19fb1487537 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:15.026794 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-cc8296d1-a709-4d01-8a46-d19fb1487537 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Getting quotas for project 528d5c15cc47445dbe46761067a1c54a. Resources: {'ram', 'instances', 'cores'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 04 01:06:15.032337 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-cc8296d1-a709-4d01-8a46-d19fb1487537 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Getting quotas for user af874cda15384e2f9160ccf6866296f6 and project 528d5c15cc47445dbe46761067a1c54a. Resources: {'ram', 'instances', 'cores'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 04 01:06:15.036836 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_api:SELECT=100 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:15.047625 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-cc8296d1-a709-4d01-8a46-d19fb1487537 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Going to run 1 instances... {{(pid=121036) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 04 01:06:15.050941 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-cc8296d1-a709-4d01-8a46-d19fb1487537 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] [instance: aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a11315c3-ce81-478b-91e4-5353f189dfa7',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121036) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 04 01:06:15.077947 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-795a4f00-aceb-49a9-8a30-92a5fd6bc0f8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:15.078543 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-795a4f00-aceb-49a9-8a30-92a5fd6bc0f8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:15.085009 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-795a4f00-aceb-49a9-8a30-92a5fd6bc0f8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:15.085854 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-795a4f00-aceb-49a9-8a30-92a5fd6bc0f8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:15.086447 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-795a4f00-aceb-49a9-8a30-92a5fd6bc0f8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:15.087310 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-795a4f00-aceb-49a9-8a30-92a5fd6bc0f8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:15.088258 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-795a4f00-aceb-49a9-8a30-92a5fd6bc0f8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:15.088828 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-795a4f00-aceb-49a9-8a30-92a5fd6bc0f8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:15.271763 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-795a4f00-aceb-49a9-8a30-92a5fd6bc0f8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:15.280456 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-795a4f00-aceb-49a9-8a30-92a5fd6bc0f8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.206212 Sep 04 01:06:15.281660 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 85/200] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:15 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:06:15.473460 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-258fd51b-4ddf-4350-8c28-501c62715925 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:15.474433 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-258fd51b-4ddf-4350-8c28-501c62715925 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:15.481022 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-258fd51b-4ddf-4350-8c28-501c62715925 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:15.481929 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-258fd51b-4ddf-4350-8c28-501c62715925 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:15.482608 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-258fd51b-4ddf-4350-8c28-501c62715925 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:15.483657 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-258fd51b-4ddf-4350-8c28-501c62715925 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:15.484436 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-258fd51b-4ddf-4350-8c28-501c62715925 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:15.485205 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-258fd51b-4ddf-4350-8c28-501c62715925 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:15.662322 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-cc8296d1-a709-4d01-8a46-d19fb1487537 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.084244 Sep 04 01:06:15.662963 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 116/201] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 4 01:06:14 2022] POST /compute/v2.1/servers => generated 380 bytes in 1086 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 04 01:06:15.675291 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-0f412311-6ead-4b5d-946b-1c976cefed96 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:15.680462 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-0f412311-6ead-4b5d-946b-1c976cefed96 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] [instance: aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:15.745855 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-0f412311-6ead-4b5d-946b-1c976cefed96 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af874cda15384e2f9160ccf6866296f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '528d5c15cc47445dbe46761067a1c54a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:15.748543 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-0f412311-6ead-4b5d-946b-1c976cefed96 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] 10.222.0.42 "GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8" status: 200 len: 1215 microversion: 2.1 time: 0.076724 Sep 04 01:06:15.749686 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 117/202] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:15 2022] GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8 => generated 1215 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:15.950656 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-258fd51b-4ddf-4350-8c28-501c62715925 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:15.959824 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-258fd51b-4ddf-4350-8c28-501c62715925 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:15.961333 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-258fd51b-4ddf-4350-8c28-501c62715925 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:15.962484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-258fd51b-4ddf-4350-8c28-501c62715925 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1597 microversion: 2.20 time: 0.494686 Sep 04 01:06:15.963044 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 86/203] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:06:15 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1597 bytes in 496 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:06:15.992161 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-ee24cdb2-836b-4ce4-a9e9-36d938090bc4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:15.992903 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-ee24cdb2-836b-4ce4-a9e9-36d938090bc4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:16.000548 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ee24cdb2-836b-4ce4-a9e9-36d938090bc4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:16.001127 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ee24cdb2-836b-4ce4-a9e9-36d938090bc4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:16.001637 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ee24cdb2-836b-4ce4-a9e9-36d938090bc4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:16.003254 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ee24cdb2-836b-4ce4-a9e9-36d938090bc4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:16.003957 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ee24cdb2-836b-4ce4-a9e9-36d938090bc4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:16.004435 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ee24cdb2-836b-4ce4-a9e9-36d938090bc4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:16.093152 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-ee24cdb2-836b-4ce4-a9e9-36d938090bc4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.104847 Sep 04 01:06:16.094295 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 118/204] 10.222.0.42 () {60 vars in 1316 bytes} [Sun Sep 4 01:06:15 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3/os-interface => generated 284 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 04 01:06:16.306434 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-631c3308-4b2e-4907-a74c-3a3f92a22973 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:16.307883 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-631c3308-4b2e-4907-a74c-3a3f92a22973 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:16.314831 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-631c3308-4b2e-4907-a74c-3a3f92a22973 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:16.316149 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-631c3308-4b2e-4907-a74c-3a3f92a22973 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:16.316777 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-631c3308-4b2e-4907-a74c-3a3f92a22973 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:16.317706 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-631c3308-4b2e-4907-a74c-3a3f92a22973 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:16.318162 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-631c3308-4b2e-4907-a74c-3a3f92a22973 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:16.318596 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-631c3308-4b2e-4907-a74c-3a3f92a22973 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:16.526056 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-631c3308-4b2e-4907-a74c-3a3f92a22973 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:16.537011 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-631c3308-4b2e-4907-a74c-3a3f92a22973 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.234430 Sep 04 01:06:16.537968 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 87/205] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:16 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:06:16.774369 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-cf786446-ec83-4055-99d8-9c246af2d02d tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:16.774969 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-cf786446-ec83-4055-99d8-9c246af2d02d tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] [instance: aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:16.864899 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-cf786446-ec83-4055-99d8-9c246af2d02d tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af874cda15384e2f9160ccf6866296f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '528d5c15cc47445dbe46761067a1c54a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:16.886575 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-cf786446-ec83-4055-99d8-9c246af2d02d tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] 10.222.0.42 "GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8" status: 200 len: 1215 microversion: 2.1 time: 0.116756 Sep 04 01:06:16.889287 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 119/206] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:16 2022] GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8 => generated 1215 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:17.499101 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-2453a2ff-31b7-4d29-aadb-24a251bcac09 req-717a166d-2192-4df5-971b-3abcfeafbf45 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "49be5a0b-f748-486e-a546-c07a8e8451a3", "tag": "fcbc60fd-1785-4f94-9436-a520ec06cdaf"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:06:17.549354 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-2453a2ff-31b7-4d29-aadb-24a251bcac09 req-717a166d-2192-4df5-971b-3abcfeafbf45 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:17.550548 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-2453a2ff-31b7-4d29-aadb-24a251bcac09 req-717a166d-2192-4df5-971b-3abcfeafbf45 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:17.551105 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-2453a2ff-31b7-4d29-aadb-24a251bcac09 req-717a166d-2192-4df5-971b-3abcfeafbf45 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:17.561682 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-425b3b27-20d2-4deb-b874-456e8c763a68 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:17.563367 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-425b3b27-20d2-4deb-b874-456e8c763a68 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:17.564735 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-2453a2ff-31b7-4d29-aadb-24a251bcac09 req-717a166d-2192-4df5-971b-3abcfeafbf45 service nova] Creating event network-changed:fcbc60fd-1785-4f94-9436-a520ec06cdaf for instance 49be5a0b-f748-486e-a546-c07a8e8451a3 on hv-cinder-maste Sep 04 01:06:17.572642 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-2453a2ff-31b7-4d29-aadb-24a251bcac09 req-717a166d-2192-4df5-971b-3abcfeafbf45 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.077847 Sep 04 01:06:17.573834 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 88/207] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:06:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:06:17.574902 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-425b3b27-20d2-4deb-b874-456e8c763a68 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:17.575408 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-425b3b27-20d2-4deb-b874-456e8c763a68 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:17.575845 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-425b3b27-20d2-4deb-b874-456e8c763a68 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:17.578666 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-425b3b27-20d2-4deb-b874-456e8c763a68 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:17.579248 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-425b3b27-20d2-4deb-b874-456e8c763a68 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:17.580283 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-425b3b27-20d2-4deb-b874-456e8c763a68 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:17.617070 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=100 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:17.750576 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-425b3b27-20d2-4deb-b874-456e8c763a68 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:17.769573 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-425b3b27-20d2-4deb-b874-456e8c763a68 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.205675 Sep 04 01:06:17.770394 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 120/208] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:17 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:17.914505 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-f176b894-a627-4eb9-a519-b9a0ec740dd1 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:17.915213 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-f176b894-a627-4eb9-a519-b9a0ec740dd1 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] [instance: aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:17.923550 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f176b894-a627-4eb9-a519-b9a0ec740dd1 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:17.924167 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f176b894-a627-4eb9-a519-b9a0ec740dd1 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:17.924872 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f176b894-a627-4eb9-a519-b9a0ec740dd1 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:17.926227 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f176b894-a627-4eb9-a519-b9a0ec740dd1 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:17.927145 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f176b894-a627-4eb9-a519-b9a0ec740dd1 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:17.928132 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f176b894-a627-4eb9-a519-b9a0ec740dd1 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:18.039179 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-f176b894-a627-4eb9-a519-b9a0ec740dd1 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af874cda15384e2f9160ccf6866296f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '528d5c15cc47445dbe46761067a1c54a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:18.050091 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-f176b894-a627-4eb9-a519-b9a0ec740dd1 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] 10.222.0.42 "GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8" status: 200 len: 1267 microversion: 2.1 time: 0.141533 Sep 04 01:06:18.051277 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 89/209] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:17 2022] GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8 => generated 1267 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:18.794210 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-12a73df5-ccc8-449c-a920-368df721c440 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:18.795009 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-12a73df5-ccc8-449c-a920-368df721c440 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:18.802143 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-12a73df5-ccc8-449c-a920-368df721c440 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:18.802747 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-12a73df5-ccc8-449c-a920-368df721c440 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:18.803464 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-12a73df5-ccc8-449c-a920-368df721c440 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:18.804266 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-12a73df5-ccc8-449c-a920-368df721c440 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:18.804824 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-12a73df5-ccc8-449c-a920-368df721c440 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:18.805463 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-12a73df5-ccc8-449c-a920-368df721c440 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:18.951890 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-12a73df5-ccc8-449c-a920-368df721c440 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:18.960288 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-12a73df5-ccc8-449c-a920-368df721c440 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.170465 Sep 04 01:06:18.960878 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 121/210] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:18 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:19.077476 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-7894fa96-0527-4806-8451-9683a6e67a26 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:19.078258 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-7894fa96-0527-4806-8451-9683a6e67a26 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] [instance: aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:19.085377 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7894fa96-0527-4806-8451-9683a6e67a26 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:19.085917 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7894fa96-0527-4806-8451-9683a6e67a26 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:19.086356 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7894fa96-0527-4806-8451-9683a6e67a26 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:19.087046 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7894fa96-0527-4806-8451-9683a6e67a26 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:19.087503 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7894fa96-0527-4806-8451-9683a6e67a26 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:19.087930 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7894fa96-0527-4806-8451-9683a6e67a26 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:19.150626 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-7894fa96-0527-4806-8451-9683a6e67a26 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af874cda15384e2f9160ccf6866296f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '528d5c15cc47445dbe46761067a1c54a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:19.158464 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-7894fa96-0527-4806-8451-9683a6e67a26 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] 10.222.0.42 "GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8" status: 200 len: 1275 microversion: 2.1 time: 0.084880 Sep 04 01:06:19.159714 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 90/211] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:19 2022] GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8 => generated 1275 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:19.983142 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-69916d0f-1049-485a-8e8d-4b8f71949dc4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:19.983747 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-69916d0f-1049-485a-8e8d-4b8f71949dc4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:19.990156 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-69916d0f-1049-485a-8e8d-4b8f71949dc4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:19.990631 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-69916d0f-1049-485a-8e8d-4b8f71949dc4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:19.991063 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-69916d0f-1049-485a-8e8d-4b8f71949dc4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:19.991745 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-69916d0f-1049-485a-8e8d-4b8f71949dc4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:19.992199 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-69916d0f-1049-485a-8e8d-4b8f71949dc4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:19.992619 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-69916d0f-1049-485a-8e8d-4b8f71949dc4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:20.117402 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-69916d0f-1049-485a-8e8d-4b8f71949dc4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:20.125055 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-69916d0f-1049-485a-8e8d-4b8f71949dc4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.145855 Sep 04 01:06:20.125817 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 122/212] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:19 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:20.183006 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-77d7a5db-ca51-4440-ad04-e923e97225a7 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:20.183613 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-77d7a5db-ca51-4440-ad04-e923e97225a7 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] [instance: aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:20.190670 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-77d7a5db-ca51-4440-ad04-e923e97225a7 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:20.191152 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-77d7a5db-ca51-4440-ad04-e923e97225a7 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:20.191583 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-77d7a5db-ca51-4440-ad04-e923e97225a7 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:20.192274 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-77d7a5db-ca51-4440-ad04-e923e97225a7 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:20.192723 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-77d7a5db-ca51-4440-ad04-e923e97225a7 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:20.193174 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-77d7a5db-ca51-4440-ad04-e923e97225a7 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:20.254072 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-77d7a5db-ca51-4440-ad04-e923e97225a7 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af874cda15384e2f9160ccf6866296f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '528d5c15cc47445dbe46761067a1c54a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:20.262066 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-77d7a5db-ca51-4440-ad04-e923e97225a7 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] 10.222.0.42 "GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8" status: 200 len: 1285 microversion: 2.1 time: 0.082886 Sep 04 01:06:20.267127 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 91/213] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:20 2022] GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8 => generated 1285 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:21.148615 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-4297757d-edde-4aca-a594-95484739f4d5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:21.152789 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-4297757d-edde-4aca-a594-95484739f4d5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:21.155870 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4297757d-edde-4aca-a594-95484739f4d5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:21.156368 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4297757d-edde-4aca-a594-95484739f4d5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:21.157037 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4297757d-edde-4aca-a594-95484739f4d5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:21.157763 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4297757d-edde-4aca-a594-95484739f4d5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:21.158807 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4297757d-edde-4aca-a594-95484739f4d5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:21.159245 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4297757d-edde-4aca-a594-95484739f4d5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:21.285353 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-e00a660f-e98c-431c-bfef-e807560d470c tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:21.285974 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-e00a660f-e98c-431c-bfef-e807560d470c tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] [instance: aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:21.293908 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-4297757d-edde-4aca-a594-95484739f4d5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:21.298232 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e00a660f-e98c-431c-bfef-e807560d470c tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:21.298667 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e00a660f-e98c-431c-bfef-e807560d470c tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:21.299106 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e00a660f-e98c-431c-bfef-e807560d470c tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:21.302044 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-4297757d-edde-4aca-a594-95484739f4d5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.157432 Sep 04 01:06:21.313139 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 123/214] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:21 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:21.313302 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e00a660f-e98c-431c-bfef-e807560d470c tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:21.313302 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e00a660f-e98c-431c-bfef-e807560d470c tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:21.313302 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e00a660f-e98c-431c-bfef-e807560d470c tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:21.383673 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-e00a660f-e98c-431c-bfef-e807560d470c tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af874cda15384e2f9160ccf6866296f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '528d5c15cc47445dbe46761067a1c54a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:21.398210 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-e00a660f-e98c-431c-bfef-e807560d470c tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] 10.222.0.42 "GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8" status: 200 len: 1285 microversion: 2.1 time: 0.116946 Sep 04 01:06:21.399633 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 92/215] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:21 2022] GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8 => generated 1285 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:06:22.326322 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-9e01b993-beea-4b9a-8bb4-5be0e4beeaad tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:22.327064 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-9e01b993-beea-4b9a-8bb4-5be0e4beeaad tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:22.334432 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9e01b993-beea-4b9a-8bb4-5be0e4beeaad tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:22.335000 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9e01b993-beea-4b9a-8bb4-5be0e4beeaad tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:22.335499 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9e01b993-beea-4b9a-8bb4-5be0e4beeaad tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:22.336303 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9e01b993-beea-4b9a-8bb4-5be0e4beeaad tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:22.336857 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9e01b993-beea-4b9a-8bb4-5be0e4beeaad tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:22.337364 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9e01b993-beea-4b9a-8bb4-5be0e4beeaad tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:22.425867 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-e8777dad-7484-42cd-bde2-050d52919e01 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:22.426256 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-e8777dad-7484-42cd-bde2-050d52919e01 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] [instance: aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:22.440745 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e8777dad-7484-42cd-bde2-050d52919e01 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:22.441657 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e8777dad-7484-42cd-bde2-050d52919e01 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:22.443812 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e8777dad-7484-42cd-bde2-050d52919e01 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:22.445619 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e8777dad-7484-42cd-bde2-050d52919e01 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:22.447187 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e8777dad-7484-42cd-bde2-050d52919e01 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:22.447711 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e8777dad-7484-42cd-bde2-050d52919e01 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:22.532854 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-9e01b993-beea-4b9a-8bb4-5be0e4beeaad tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:22.544099 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-9e01b993-beea-4b9a-8bb4-5be0e4beeaad tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.221498 Sep 04 01:06:22.545156 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 124/216] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:22 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:22.670957 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-e8777dad-7484-42cd-bde2-050d52919e01 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af874cda15384e2f9160ccf6866296f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '528d5c15cc47445dbe46761067a1c54a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:22.678931 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-e8777dad-7484-42cd-bde2-050d52919e01 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] 10.222.0.42 "GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8" status: 200 len: 1315 microversion: 2.1 time: 0.259801 Sep 04 01:06:22.679775 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 93/217] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:22 2022] GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8 => generated 1315 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:23.070431 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-df611cbb-7aa5-479f-a4db-d7d00679a2c8 req-a805e651-0698-4013-abca-fd3cd5c3bcbf service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8", "tag": "0af19f52-43e4-40ca-848a-d2902c3e8205"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:06:23.079837 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-df611cbb-7aa5-479f-a4db-d7d00679a2c8 req-a805e651-0698-4013-abca-fd3cd5c3bcbf service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:23.080314 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-df611cbb-7aa5-479f-a4db-d7d00679a2c8 req-a805e651-0698-4013-abca-fd3cd5c3bcbf service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:23.080744 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-df611cbb-7aa5-479f-a4db-d7d00679a2c8 req-a805e651-0698-4013-abca-fd3cd5c3bcbf service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:23.091914 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-df611cbb-7aa5-479f-a4db-d7d00679a2c8 req-a805e651-0698-4013-abca-fd3cd5c3bcbf service nova] Creating event network-changed:0af19f52-43e4-40ca-848a-d2902c3e8205 for instance aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8 on hv-cinder-maste Sep 04 01:06:23.101027 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-df611cbb-7aa5-479f-a4db-d7d00679a2c8 req-a805e651-0698-4013-abca-fd3cd5c3bcbf service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035695 Sep 04 01:06:23.101656 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 125/218] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:06:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:06:23.268745 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-de3af377-f086-424c-863e-51322360da62 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:23.269306 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-de3af377-f086-424c-863e-51322360da62 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:23.275155 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-de3af377-f086-424c-863e-51322360da62 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:23.275637 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-de3af377-f086-424c-863e-51322360da62 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:23.276065 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-de3af377-f086-424c-863e-51322360da62 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:23.276752 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-de3af377-f086-424c-863e-51322360da62 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:23.277224 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-de3af377-f086-424c-863e-51322360da62 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:23.277643 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-de3af377-f086-424c-863e-51322360da62 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:23.304233 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-de3af377-f086-424c-863e-51322360da62 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Going to try to terminate instance {{(pid=121037) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 04 01:06:23.572246 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b489ff29-f298-42fd-b645-566eedc02603 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:23.572957 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-b489ff29-f298-42fd-b645-566eedc02603 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:23.584558 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b489ff29-f298-42fd-b645-566eedc02603 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:23.584558 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b489ff29-f298-42fd-b645-566eedc02603 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:23.584558 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b489ff29-f298-42fd-b645-566eedc02603 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:23.584558 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b489ff29-f298-42fd-b645-566eedc02603 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:23.584870 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b489ff29-f298-42fd-b645-566eedc02603 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:23.584870 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b489ff29-f298-42fd-b645-566eedc02603 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:23.671954 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-de3af377-f086-424c-863e-51322360da62 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "DELETE /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 204 len: 0 microversion: 2.1 time: 0.406962 Sep 04 01:06:23.676157 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 94/219] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 4 01:06:23 2022] DELETE /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 04 01:06:23.685097 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-1c7cc69f-d93a-457a-a556-828ded7968b7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:23.686156 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-1c7cc69f-d93a-457a-a556-828ded7968b7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:23.695913 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1c7cc69f-d93a-457a-a556-828ded7968b7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:23.696930 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1c7cc69f-d93a-457a-a556-828ded7968b7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:23.697698 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1c7cc69f-d93a-457a-a556-828ded7968b7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:23.704860 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1c7cc69f-d93a-457a-a556-828ded7968b7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:23.704860 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1c7cc69f-d93a-457a-a556-828ded7968b7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:23.704860 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1c7cc69f-d93a-457a-a556-828ded7968b7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:23.755601 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-b489ff29-f298-42fd-b645-566eedc02603 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:23.763895 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-b489ff29-f298-42fd-b645-566eedc02603 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.195702 Sep 04 01:06:23.775045 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 126/220] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:23 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:23.775045 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-5265c700-c364-4e19-bbbd-fd7aa2dac48f tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:23.775045 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-5265c700-c364-4e19-bbbd-fd7aa2dac48f tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] [instance: aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:23.782935 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5265c700-c364-4e19-bbbd-fd7aa2dac48f tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:23.783576 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5265c700-c364-4e19-bbbd-fd7aa2dac48f tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:23.784591 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5265c700-c364-4e19-bbbd-fd7aa2dac48f tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:23.785407 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5265c700-c364-4e19-bbbd-fd7aa2dac48f tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:23.785943 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5265c700-c364-4e19-bbbd-fd7aa2dac48f tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:23.786504 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5265c700-c364-4e19-bbbd-fd7aa2dac48f tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:23.852701 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-1c7cc69f-d93a-457a-a556-828ded7968b7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:23.861467 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-1c7cc69f-d93a-457a-a556-828ded7968b7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1554 microversion: 2.1 time: 0.180810 Sep 04 01:06:23.863173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 95/221] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:23 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1554 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:06:23.931633 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-5265c700-c364-4e19-bbbd-fd7aa2dac48f tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af874cda15384e2f9160ccf6866296f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '528d5c15cc47445dbe46761067a1c54a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:23.939772 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-5265c700-c364-4e19-bbbd-fd7aa2dac48f tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] 10.222.0.42 "GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8" status: 200 len: 1315 microversion: 2.1 time: 0.173734 Sep 04 01:06:23.940674 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 127/222] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:23 2022] GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8 => generated 1315 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:24.790238 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-d8be257b-16dc-4a6d-ab6f-a2871938f6c0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:24.790845 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-d8be257b-16dc-4a6d-ab6f-a2871938f6c0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:24.797318 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d8be257b-16dc-4a6d-ab6f-a2871938f6c0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:24.797853 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d8be257b-16dc-4a6d-ab6f-a2871938f6c0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:24.798300 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d8be257b-16dc-4a6d-ab6f-a2871938f6c0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:24.799062 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d8be257b-16dc-4a6d-ab6f-a2871938f6c0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:24.800166 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d8be257b-16dc-4a6d-ab6f-a2871938f6c0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:24.800605 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d8be257b-16dc-4a6d-ab6f-a2871938f6c0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:24.815346 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:24.884708 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-34ed4ef6-5e7f-4ac6-bcfe-7f4e48f9f99d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:24.885339 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-34ed4ef6-5e7f-4ac6-bcfe-7f4e48f9f99d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:24.891448 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-34ed4ef6-5e7f-4ac6-bcfe-7f4e48f9f99d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:24.891825 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-34ed4ef6-5e7f-4ac6-bcfe-7f4e48f9f99d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:24.892249 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-34ed4ef6-5e7f-4ac6-bcfe-7f4e48f9f99d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:24.892939 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-34ed4ef6-5e7f-4ac6-bcfe-7f4e48f9f99d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:24.893419 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-34ed4ef6-5e7f-4ac6-bcfe-7f4e48f9f99d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:24.893902 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-34ed4ef6-5e7f-4ac6-bcfe-7f4e48f9f99d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:24.931247 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-d8be257b-16dc-4a6d-ab6f-a2871938f6c0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:24.939122 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-d8be257b-16dc-4a6d-ab6f-a2871938f6c0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.152786 Sep 04 01:06:24.939798 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 96/223] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:24 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:06:24.966859 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-ba519a93-7de9-4d59-a86e-fb9fe595cc56 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:24.967572 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-ba519a93-7de9-4d59-a86e-fb9fe595cc56 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] [instance: aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:24.974468 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ba519a93-7de9-4d59-a86e-fb9fe595cc56 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:24.974937 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ba519a93-7de9-4d59-a86e-fb9fe595cc56 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:24.975364 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ba519a93-7de9-4d59-a86e-fb9fe595cc56 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:24.976045 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ba519a93-7de9-4d59-a86e-fb9fe595cc56 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:24.976510 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ba519a93-7de9-4d59-a86e-fb9fe595cc56 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:24.976902 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ba519a93-7de9-4d59-a86e-fb9fe595cc56 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:25.017134 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell0:SELECT=2 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:25.025637 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-34ed4ef6-5e7f-4ac6-bcfe-7f4e48f9f99d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:25.034292 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-34ed4ef6-5e7f-4ac6-bcfe-7f4e48f9f99d tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1554 microversion: 2.1 time: 0.153159 Sep 04 01:06:25.035069 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 128/224] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:24 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1554 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:06:25.104642 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-ba519a93-7de9-4d59-a86e-fb9fe595cc56 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af874cda15384e2f9160ccf6866296f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '528d5c15cc47445dbe46761067a1c54a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:25.112641 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-ba519a93-7de9-4d59-a86e-fb9fe595cc56 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] 10.222.0.42 "GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8" status: 200 len: 1315 microversion: 2.1 time: 0.149768 Sep 04 01:06:25.113229 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 97/225] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:24 2022] GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8 => generated 1315 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:06:25.962145 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-be065f34-8758-41df-8501-7fc893f8e884 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:25.962748 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-be065f34-8758-41df-8501-7fc893f8e884 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:25.969387 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-be065f34-8758-41df-8501-7fc893f8e884 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:25.969864 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-be065f34-8758-41df-8501-7fc893f8e884 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:25.970337 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-be065f34-8758-41df-8501-7fc893f8e884 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:25.971053 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-be065f34-8758-41df-8501-7fc893f8e884 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:25.971552 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-be065f34-8758-41df-8501-7fc893f8e884 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:25.972087 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-be065f34-8758-41df-8501-7fc893f8e884 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:26.056938 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-b6e681c3-8ee5-4e55-8d41-289246e7d7aa tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:26.058058 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-b6e681c3-8ee5-4e55-8d41-289246e7d7aa tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:26.064466 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b6e681c3-8ee5-4e55-8d41-289246e7d7aa tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:26.065695 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b6e681c3-8ee5-4e55-8d41-289246e7d7aa tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:26.066308 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b6e681c3-8ee5-4e55-8d41-289246e7d7aa tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:26.067002 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b6e681c3-8ee5-4e55-8d41-289246e7d7aa tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:26.067602 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b6e681c3-8ee5-4e55-8d41-289246e7d7aa tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:26.068332 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b6e681c3-8ee5-4e55-8d41-289246e7d7aa tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:26.115475 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-be065f34-8758-41df-8501-7fc893f8e884 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:26.123435 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-be065f34-8758-41df-8501-7fc893f8e884 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.165197 Sep 04 01:06:26.124005 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 129/226] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:25 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:26.135361 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-1d77235c-0ba5-4077-9745-ccee688d7440 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:26.136108 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-1d77235c-0ba5-4077-9745-ccee688d7440 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] [instance: aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:26.142210 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1d77235c-0ba5-4077-9745-ccee688d7440 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:26.143412 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1d77235c-0ba5-4077-9745-ccee688d7440 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:26.144080 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1d77235c-0ba5-4077-9745-ccee688d7440 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:26.144804 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1d77235c-0ba5-4077-9745-ccee688d7440 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:26.145301 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1d77235c-0ba5-4077-9745-ccee688d7440 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:26.148453 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1d77235c-0ba5-4077-9745-ccee688d7440 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:26.212742 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-b6e681c3-8ee5-4e55-8d41-289246e7d7aa tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:26.221911 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-b6e681c3-8ee5-4e55-8d41-289246e7d7aa tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1554 microversion: 2.1 time: 0.168610 Sep 04 01:06:26.222715 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 98/227] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:26 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1554 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:06:26.280692 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-1d77235c-0ba5-4077-9745-ccee688d7440 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af874cda15384e2f9160ccf6866296f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '528d5c15cc47445dbe46761067a1c54a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:26.289106 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-1d77235c-0ba5-4077-9745-ccee688d7440 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] 10.222.0.42 "GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8" status: 200 len: 1315 microversion: 2.1 time: 0.157432 Sep 04 01:06:26.289674 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 130/228] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:26 2022] GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8 => generated 1315 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:27.147007 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-ffa59c55-c5e1-4ead-9915-c1672c94e0d7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:27.147676 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-ffa59c55-c5e1-4ead-9915-c1672c94e0d7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:27.150019 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_api:SELECT=66,nova_api:UPDATE=1 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:27.156985 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ffa59c55-c5e1-4ead-9915-c1672c94e0d7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:27.156985 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ffa59c55-c5e1-4ead-9915-c1672c94e0d7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:27.156985 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ffa59c55-c5e1-4ead-9915-c1672c94e0d7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:27.156985 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ffa59c55-c5e1-4ead-9915-c1672c94e0d7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:27.157347 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ffa59c55-c5e1-4ead-9915-c1672c94e0d7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:27.157481 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ffa59c55-c5e1-4ead-9915-c1672c94e0d7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:27.251360 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-2a06966a-2239-42ef-ad59-371cc56c9fb0 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:27.252471 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-2a06966a-2239-42ef-ad59-371cc56c9fb0 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:27.254425 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell0:SELECT=10 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:27.259353 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2a06966a-2239-42ef-ad59-371cc56c9fb0 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:27.259838 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2a06966a-2239-42ef-ad59-371cc56c9fb0 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:27.260269 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2a06966a-2239-42ef-ad59-371cc56c9fb0 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:27.260955 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2a06966a-2239-42ef-ad59-371cc56c9fb0 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:27.261590 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2a06966a-2239-42ef-ad59-371cc56c9fb0 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:27.262187 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2a06966a-2239-42ef-ad59-371cc56c9fb0 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:27.287141 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-ffa59c55-c5e1-4ead-9915-c1672c94e0d7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:27.294833 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-ffa59c55-c5e1-4ead-9915-c1672c94e0d7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.152796 Sep 04 01:06:27.295404 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 99/229] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:27 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:27.314335 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-161e663c-9fee-499f-889f-4817315842f0 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:27.315534 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-161e663c-9fee-499f-889f-4817315842f0 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] [instance: aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:27.321619 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-161e663c-9fee-499f-889f-4817315842f0 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:27.322165 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-161e663c-9fee-499f-889f-4817315842f0 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:27.322664 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-161e663c-9fee-499f-889f-4817315842f0 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:27.323414 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-161e663c-9fee-499f-889f-4817315842f0 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:27.323933 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-161e663c-9fee-499f-889f-4817315842f0 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:27.324358 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-161e663c-9fee-499f-889f-4817315842f0 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:27.420929 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-2a06966a-2239-42ef-ad59-371cc56c9fb0 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:27.435451 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-2a06966a-2239-42ef-ad59-371cc56c9fb0 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1554 microversion: 2.1 time: 0.187904 Sep 04 01:06:27.441201 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 131/230] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:27 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1554 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:06:27.595018 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-161e663c-9fee-499f-889f-4817315842f0 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af874cda15384e2f9160ccf6866296f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '528d5c15cc47445dbe46761067a1c54a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:27.602774 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-161e663c-9fee-499f-889f-4817315842f0 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] 10.222.0.42 "GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8" status: 200 len: 1315 microversion: 2.1 time: 0.292503 Sep 04 01:06:27.603343 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 100/231] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:27 2022] GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8 => generated 1315 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:28.295723 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-2293d774-b2c8-4edb-8b20-a3f5c410ffff req-25141c16-fe36-4716-a762-55873728aa67 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "742cb4b8-f03c-4256-bb96-fc2cfff43734", "name": "network-vif-deleted", "tag": "5d2b3e83-c7f4-4359-8491-abf0d6e3eec3"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:06:28.306108 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-2293d774-b2c8-4edb-8b20-a3f5c410ffff req-25141c16-fe36-4716-a762-55873728aa67 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:28.306589 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-2293d774-b2c8-4edb-8b20-a3f5c410ffff req-25141c16-fe36-4716-a762-55873728aa67 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:28.307021 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-2293d774-b2c8-4edb-8b20-a3f5c410ffff req-25141c16-fe36-4716-a762-55873728aa67 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:28.320300 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-d6414ae2-3260-4ec6-8d9a-addd6b11107a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:28.320976 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-2293d774-b2c8-4edb-8b20-a3f5c410ffff req-25141c16-fe36-4716-a762-55873728aa67 service nova] Creating event network-vif-deleted:5d2b3e83-c7f4-4359-8491-abf0d6e3eec3 for instance 742cb4b8-f03c-4256-bb96-fc2cfff43734 on hv-cinder-maste Sep 04 01:06:28.321120 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-d6414ae2-3260-4ec6-8d9a-addd6b11107a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:28.326211 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-2293d774-b2c8-4edb-8b20-a3f5c410ffff req-25141c16-fe36-4716-a762-55873728aa67 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034679 Sep 04 01:06:28.327183 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 132/232] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:06:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:06:28.333147 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d6414ae2-3260-4ec6-8d9a-addd6b11107a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:28.333633 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d6414ae2-3260-4ec6-8d9a-addd6b11107a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:28.334068 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d6414ae2-3260-4ec6-8d9a-addd6b11107a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:28.334847 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d6414ae2-3260-4ec6-8d9a-addd6b11107a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:28.335296 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d6414ae2-3260-4ec6-8d9a-addd6b11107a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:28.335719 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d6414ae2-3260-4ec6-8d9a-addd6b11107a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:28.459144 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-18225326-6fd5-4cda-85fb-915e85b05b4a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:28.459943 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-18225326-6fd5-4cda-85fb-915e85b05b4a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:28.466566 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-18225326-6fd5-4cda-85fb-915e85b05b4a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:28.467121 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-18225326-6fd5-4cda-85fb-915e85b05b4a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:28.467799 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-18225326-6fd5-4cda-85fb-915e85b05b4a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:28.468644 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-18225326-6fd5-4cda-85fb-915e85b05b4a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:28.472120 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-18225326-6fd5-4cda-85fb-915e85b05b4a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:28.472120 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-18225326-6fd5-4cda-85fb-915e85b05b4a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:28.488011 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-d6414ae2-3260-4ec6-8d9a-addd6b11107a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:28.507413 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-d6414ae2-3260-4ec6-8d9a-addd6b11107a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.186460 Sep 04 01:06:28.508943 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 101/233] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:28 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:06:28.560249 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-18225326-6fd5-4cda-85fb-915e85b05b4a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:28.569032 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-18225326-6fd5-4cda-85fb-915e85b05b4a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1345 microversion: 2.1 time: 0.115017 Sep 04 01:06:28.569632 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 133/234] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:28 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1345 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:06:28.625912 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-7d6c761c-87d8-4be6-a102-c7e53adc4b31 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:28.626511 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-7d6c761c-87d8-4be6-a102-c7e53adc4b31 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] [instance: aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:28.632414 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7d6c761c-87d8-4be6-a102-c7e53adc4b31 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:28.632887 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7d6c761c-87d8-4be6-a102-c7e53adc4b31 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:28.633359 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7d6c761c-87d8-4be6-a102-c7e53adc4b31 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:28.634039 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7d6c761c-87d8-4be6-a102-c7e53adc4b31 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:28.634480 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7d6c761c-87d8-4be6-a102-c7e53adc4b31 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:28.634895 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7d6c761c-87d8-4be6-a102-c7e53adc4b31 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:28.767267 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-7d6c761c-87d8-4be6-a102-c7e53adc4b31 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af874cda15384e2f9160ccf6866296f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '528d5c15cc47445dbe46761067a1c54a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:28.775219 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-7d6c761c-87d8-4be6-a102-c7e53adc4b31 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] 10.222.0.42 "GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8" status: 200 len: 1315 microversion: 2.1 time: 0.153092 Sep 04 01:06:28.775998 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 102/235] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:28 2022] GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8 => generated 1315 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:29.530955 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-68a8ccdf-d067-410e-bbc4-da51e395ade5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:29.535455 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-68a8ccdf-d067-410e-bbc4-da51e395ade5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:29.538241 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-68a8ccdf-d067-410e-bbc4-da51e395ade5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:29.538723 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-68a8ccdf-d067-410e-bbc4-da51e395ade5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:29.539153 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-68a8ccdf-d067-410e-bbc4-da51e395ade5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:29.539913 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-68a8ccdf-d067-410e-bbc4-da51e395ade5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:29.540418 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-68a8ccdf-d067-410e-bbc4-da51e395ade5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:29.540853 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-68a8ccdf-d067-410e-bbc4-da51e395ade5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:29.596573 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-89d43d2b-f502-4ff8-9cdc-92129b388d7a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:29.597353 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-89d43d2b-f502-4ff8-9cdc-92129b388d7a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:29.603960 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-89d43d2b-f502-4ff8-9cdc-92129b388d7a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:29.604445 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-89d43d2b-f502-4ff8-9cdc-92129b388d7a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:29.604883 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-89d43d2b-f502-4ff8-9cdc-92129b388d7a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:29.605604 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-89d43d2b-f502-4ff8-9cdc-92129b388d7a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:29.606058 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-89d43d2b-f502-4ff8-9cdc-92129b388d7a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:29.606483 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-89d43d2b-f502-4ff8-9cdc-92129b388d7a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:29.644725 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-89d43d2b-f502-4ff8-9cdc-92129b388d7a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:29.644725 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-89d43d2b-f502-4ff8-9cdc-92129b388d7a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:29.644725 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-89d43d2b-f502-4ff8-9cdc-92129b388d7a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:29.644725 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.objects.instance [None req-89d43d2b-f502-4ff8-9cdc-92129b388d7a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lazy-loading 'fault' on Instance uuid 742cb4b8-f03c-4256-bb96-fc2cfff43734 {{(pid=121037) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:06:29.681232 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-68a8ccdf-d067-410e-bbc4-da51e395ade5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:29.691275 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-68a8ccdf-d067-410e-bbc4-da51e395ade5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.164030 Sep 04 01:06:29.691937 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 134/236] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:29 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:29.695610 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-89d43d2b-f502-4ff8-9cdc-92129b388d7a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'b6b95b6814cf4cd8b77d87e7810c70e5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e07d8b14fef34e04a0410f12e5dbd2fe', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:29.711997 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-89d43d2b-f502-4ff8-9cdc-92129b388d7a tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 200 len: 1350 microversion: 2.1 time: 0.121828 Sep 04 01:06:29.712679 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 103/237] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:29 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 1350 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:29.801011 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-497a84d0-d08e-43e3-a19c-48cebf6f7b4c tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:29.801741 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-497a84d0-d08e-43e3-a19c-48cebf6f7b4c tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] [instance: aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:29.807704 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-497a84d0-d08e-43e3-a19c-48cebf6f7b4c tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:29.808262 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-497a84d0-d08e-43e3-a19c-48cebf6f7b4c tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:29.808710 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-497a84d0-d08e-43e3-a19c-48cebf6f7b4c tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:29.809423 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-497a84d0-d08e-43e3-a19c-48cebf6f7b4c tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:29.809876 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-497a84d0-d08e-43e3-a19c-48cebf6f7b4c tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:29.810297 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-497a84d0-d08e-43e3-a19c-48cebf6f7b4c tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:29.975271 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-497a84d0-d08e-43e3-a19c-48cebf6f7b4c tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af874cda15384e2f9160ccf6866296f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '528d5c15cc47445dbe46761067a1c54a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:29.984296 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-497a84d0-d08e-43e3-a19c-48cebf6f7b4c tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] 10.222.0.42 "GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8" status: 200 len: 1315 microversion: 2.1 time: 0.187136 Sep 04 01:06:29.985222 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 135/238] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:29 2022] GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8 => generated 1315 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:30.717502 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-b6843791-1478-49d7-a4ad-e85ce469128f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:30.718808 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-b6843791-1478-49d7-a4ad-e85ce469128f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:30.728268 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b6843791-1478-49d7-a4ad-e85ce469128f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:30.728858 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b6843791-1478-49d7-a4ad-e85ce469128f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:30.729538 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b6843791-1478-49d7-a4ad-e85ce469128f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:30.730541 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b6843791-1478-49d7-a4ad-e85ce469128f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:30.731222 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b6843791-1478-49d7-a4ad-e85ce469128f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:30.731853 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b6843791-1478-49d7-a4ad-e85ce469128f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:30.736934 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-cbbcd2eb-8102-43e8-9074-15ffa424c92c tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:30.740056 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-cbbcd2eb-8102-43e8-9074-15ffa424c92c tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:30.744827 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cbbcd2eb-8102-43e8-9074-15ffa424c92c tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:30.745363 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cbbcd2eb-8102-43e8-9074-15ffa424c92c tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:30.745801 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cbbcd2eb-8102-43e8-9074-15ffa424c92c tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:30.746508 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cbbcd2eb-8102-43e8-9074-15ffa424c92c tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:30.746963 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cbbcd2eb-8102-43e8-9074-15ffa424c92c tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:30.747389 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cbbcd2eb-8102-43e8-9074-15ffa424c92c tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:30.901686 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.wsgi [None req-cbbcd2eb-8102-43e8-9074-15ffa424c92c tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] HTTP exception thrown: Instance 742cb4b8-f03c-4256-bb96-fc2cfff43734 could not be found. Sep 04 01:06:30.906599 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-cbbcd2eb-8102-43e8-9074-15ffa424c92c tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Request method failure captured: Sep 04 01:06:30.906599 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: request: GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 HTTP/1.1 Sep 04 01:06:30.906599 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept: application/json Sep 04 01:06:30.906599 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept-Encoding: identity Sep 04 01:06:30.906599 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Connection: close Sep 04 01:06:30.906599 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Length: 0 Sep 04 01:06:30.906599 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Type: application/json Sep 04 01:06:30.906599 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Host: 10.222.0.42 Sep 04 01:06:30.906599 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Openstack-System-Scope: None Sep 04 01:06:30.906599 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: User-Agent: python-urllib3/1.26.12 Sep 04 01:06:30.906599 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Auth-Token: gAAAAABjE_nDnngl7ftGavz-i2_6lnN2Oc2VvKaTiT-mXQQYKbwql02Y_PHxrmOsik8amb4rncT4NZZw0SrQ9gjmjUjdvAcquMu5ExeRYTuFG3mA6_Urv181U-EXuCZS1rfVDhwXABPfhIeZjcFWrOd62o-R4_0qHvBkErxliTR8plS61B0SPYs Sep 04 01:06:30.906599 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Id: None Sep 04 01:06:30.906599 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Name: None Sep 04 01:06:30.906599 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Identity-Status: Confirmed Sep 04 01:06:30.906599 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Is-Admin-Project: True Sep 04 01:06:30.906599 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Id: default Sep 04 01:06:30.906599 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Name: Default Sep 04 01:06:30.906599 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Id: e07d8b14fef34e04a0410f12e5dbd2fe Sep 04 01:06:30.906599 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Name: tempest-DeleteServersTestJSON-1716711868 Sep 04 01:06:30.906599 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Role: reader,member Sep 04 01:06:30.906599 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Roles: reader,member Sep 04 01:06:30.907930 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/e07d8b14fef34e04a0410f12e5dbd2fe"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_e07d8b14fef34e04a0410f12e5dbd2fe", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/e07d8b14fef34e04a0410f12e5dbd2fe"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/e07d8b14fef34e04a0410f12e5dbd2fe"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:06:30.907930 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant: tempest-DeleteServersTestJSON-1716711868 Sep 04 01:06:30.907930 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Id: e07d8b14fef34e04a0410f12e5dbd2fe Sep 04 01:06:30.907930 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Name: tempest-DeleteServersTestJSON-1716711868 Sep 04 01:06:30.907930 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User: tempest-DeleteServersTestJSON-1716711868-project Sep 04 01:06:30.907930 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Id: default Sep 04 01:06:30.907930 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Name: Default Sep 04 01:06:30.907930 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Id: b6b95b6814cf4cd8b77d87e7810c70e5 Sep 04 01:06:30.907930 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Name: tempest-DeleteServersTestJSON-1716711868-project Sep 04 01:06:30.907930 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: method: > Sep 04 01:06:30.907930 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: exception: Instance 742cb4b8-f03c-4256-bb96-fc2cfff43734 could not be found. Sep 04 01:06:30.907930 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: action_args: {'id': '742cb4b8-f03c-4256-bb96-fc2cfff43734'} Sep 04 01:06:30.908671 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:06:30.908671 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:06:30.908671 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:06:30.908671 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:06:30.908671 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:06:30.908671 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:06:30.908671 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:06:30.908671 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:06:30.908671 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:06:30.908671 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:06:30.908671 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 742cb4b8-f03c-4256-bb96-fc2cfff43734 could not be found. Sep 04 01:06:30.908671 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:06:30.908671 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:06:30.908671 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:06:30.908671 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:06:30.908671 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:06:30.908671 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:06:30.908671 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:06:30.908671 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:06:30.908671 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:06:30.915604 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:06:30.915604 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 04 01:06:30.915604 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 04 01:06:30.915604 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:06:30.915604 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:06:30.915604 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:06:30.915604 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:06:30.915604 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 742cb4b8-f03c-4256-bb96-fc2cfff43734 could not be found. Sep 04 01:06:30.915604 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:06:30.915604 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:06:30.915604 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:06:30.915604 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:06:30.915604 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:06:30.915604 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:06:30.915604 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:06:30.915604 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:06:30.915604 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 742cb4b8-f03c-4256-bb96-fc2cfff43734 could not be found. Sep 04 01:06:30.915604 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:06:30.917165 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-cbbcd2eb-8102-43e8-9074-15ffa424c92c tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Returning 404 to user: Instance 742cb4b8-f03c-4256-bb96-fc2cfff43734 could not be found. {{(pid=121036) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:06:30.917165 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-cbbcd2eb-8102-43e8-9074-15ffa424c92c tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 404 len: 111 microversion: 2.1 time: 0.176270 Sep 04 01:06:30.917165 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 136/239] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:30 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 111 bytes in 177 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 04 01:06:30.948800 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-b6843791-1478-49d7-a4ad-e85ce469128f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:30.959359 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-b6843791-1478-49d7-a4ad-e85ce469128f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.247151 Sep 04 01:06:30.959947 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 104/240] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:30 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:31.008496 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-7d8c01b5-2cb4-44b8-a310-0146631f15cb tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:31.009128 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-7d8c01b5-2cb4-44b8-a310-0146631f15cb tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] [instance: aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:31.015024 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7d8c01b5-2cb4-44b8-a310-0146631f15cb tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:31.015704 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7d8c01b5-2cb4-44b8-a310-0146631f15cb tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:31.016102 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7d8c01b5-2cb4-44b8-a310-0146631f15cb tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:31.016783 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7d8c01b5-2cb4-44b8-a310-0146631f15cb tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:31.017253 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7d8c01b5-2cb4-44b8-a310-0146631f15cb tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:31.017827 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7d8c01b5-2cb4-44b8-a310-0146631f15cb tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:31.050270 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-553d0bb9-6bd2-4f87-973a-fd70eb1191b7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:31.059697 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-553d0bb9-6bd2-4f87-973a-fd70eb1191b7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:31.066944 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-553d0bb9-6bd2-4f87-973a-fd70eb1191b7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:31.067365 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-553d0bb9-6bd2-4f87-973a-fd70eb1191b7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:31.067872 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-553d0bb9-6bd2-4f87-973a-fd70eb1191b7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:31.069409 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-553d0bb9-6bd2-4f87-973a-fd70eb1191b7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:31.070232 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-553d0bb9-6bd2-4f87-973a-fd70eb1191b7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:31.070819 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-553d0bb9-6bd2-4f87-973a-fd70eb1191b7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:31.101061 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.wsgi [None req-553d0bb9-6bd2-4f87-973a-fd70eb1191b7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] HTTP exception thrown: Instance 742cb4b8-f03c-4256-bb96-fc2cfff43734 could not be found. Sep 04 01:06:31.110235 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-553d0bb9-6bd2-4f87-973a-fd70eb1191b7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Request method failure captured: Sep 04 01:06:31.110235 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: request: GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734/os-volume_attachments HTTP/1.1 Sep 04 01:06:31.110235 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept: application/json Sep 04 01:06:31.110235 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept-Encoding: identity Sep 04 01:06:31.110235 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Connection: close Sep 04 01:06:31.110235 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Length: 0 Sep 04 01:06:31.110235 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Type: application/json Sep 04 01:06:31.110235 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Host: 10.222.0.42 Sep 04 01:06:31.110235 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Openstack-System-Scope: None Sep 04 01:06:31.110235 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: User-Agent: python-urllib3/1.26.12 Sep 04 01:06:31.110235 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Auth-Token: gAAAAABjE_nDnngl7ftGavz-i2_6lnN2Oc2VvKaTiT-mXQQYKbwql02Y_PHxrmOsik8amb4rncT4NZZw0SrQ9gjmjUjdvAcquMu5ExeRYTuFG3mA6_Urv181U-EXuCZS1rfVDhwXABPfhIeZjcFWrOd62o-R4_0qHvBkErxliTR8plS61B0SPYs Sep 04 01:06:31.110235 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Id: None Sep 04 01:06:31.110235 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Name: None Sep 04 01:06:31.110235 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Identity-Status: Confirmed Sep 04 01:06:31.110235 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Is-Admin-Project: True Sep 04 01:06:31.110235 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Id: default Sep 04 01:06:31.110235 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Name: Default Sep 04 01:06:31.110235 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Id: e07d8b14fef34e04a0410f12e5dbd2fe Sep 04 01:06:31.110235 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Name: tempest-DeleteServersTestJSON-1716711868 Sep 04 01:06:31.110235 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Role: reader,member Sep 04 01:06:31.110235 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Roles: reader,member Sep 04 01:06:31.111725 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/e07d8b14fef34e04a0410f12e5dbd2fe"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_e07d8b14fef34e04a0410f12e5dbd2fe", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/e07d8b14fef34e04a0410f12e5dbd2fe"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/e07d8b14fef34e04a0410f12e5dbd2fe"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:06:31.111725 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant: tempest-DeleteServersTestJSON-1716711868 Sep 04 01:06:31.111725 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Id: e07d8b14fef34e04a0410f12e5dbd2fe Sep 04 01:06:31.111725 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Name: tempest-DeleteServersTestJSON-1716711868 Sep 04 01:06:31.111725 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User: tempest-DeleteServersTestJSON-1716711868-project Sep 04 01:06:31.111725 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Id: default Sep 04 01:06:31.111725 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Name: Default Sep 04 01:06:31.111725 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Id: b6b95b6814cf4cd8b77d87e7810c70e5 Sep 04 01:06:31.111725 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Name: tempest-DeleteServersTestJSON-1716711868-project Sep 04 01:06:31.111725 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: method: > Sep 04 01:06:31.111725 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: exception: Instance 742cb4b8-f03c-4256-bb96-fc2cfff43734 could not be found. Sep 04 01:06:31.111725 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: action_args: {'server_id': '742cb4b8-f03c-4256-bb96-fc2cfff43734'} Sep 04 01:06:31.112406 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:06:31.112406 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:06:31.112406 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:06:31.112406 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:06:31.112406 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:06:31.112406 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:06:31.112406 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:06:31.112406 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:06:31.112406 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:06:31.112406 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:06:31.112406 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 742cb4b8-f03c-4256-bb96-fc2cfff43734 could not be found. Sep 04 01:06:31.112406 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:06:31.112406 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:06:31.112406 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:06:31.112406 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:06:31.112406 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:06:31.112406 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:06:31.112406 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:06:31.112406 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:06:31.112406 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:06:31.118121 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:06:31.118121 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 04 01:06:31.118121 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 04 01:06:31.118121 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 04 01:06:31.118121 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 04 01:06:31.118121 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Sep 04 01:06:31.118121 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Sep 04 01:06:31.118121 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:06:31.118121 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:06:31.118121 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 742cb4b8-f03c-4256-bb96-fc2cfff43734 could not be found. Sep 04 01:06:31.118121 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:06:31.118121 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:06:31.118121 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:06:31.118121 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:06:31.118121 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:06:31.118121 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:06:31.118121 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:06:31.118121 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:06:31.118121 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 742cb4b8-f03c-4256-bb96-fc2cfff43734 could not be found. Sep 04 01:06:31.118121 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:06:31.125981 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-553d0bb9-6bd2-4f87-973a-fd70eb1191b7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Returning 404 to user: Instance 742cb4b8-f03c-4256-bb96-fc2cfff43734 could not be found. {{(pid=121037) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:06:31.125981 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-553d0bb9-6bd2-4f87-973a-fd70eb1191b7 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.065776 Sep 04 01:06:31.125981 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 105/241] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 4 01:06:31 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734/os-volume_attachments => generated 111 bytes in 70 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 04 01:06:31.169583 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-7d8c01b5-2cb4-44b8-a310-0146631f15cb tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af874cda15384e2f9160ccf6866296f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '528d5c15cc47445dbe46761067a1c54a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:31.181876 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-7d8c01b5-2cb4-44b8-a310-0146631f15cb tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] 10.222.0.42 "GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8" status: 200 len: 1315 microversion: 2.1 time: 0.177075 Sep 04 01:06:31.183709 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 137/242] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:31 2022] GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8 => generated 1315 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:31.261266 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-01b273ae-3130-44fc-985a-6a426d65531e req-42fde18b-2688-4299-8494-4b62ebca5a04 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8", "name": "network-vif-plugged", "status": "completed", "tag": "0af19f52-43e4-40ca-848a-d2902c3e8205"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:06:31.271869 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-01b273ae-3130-44fc-985a-6a426d65531e req-42fde18b-2688-4299-8494-4b62ebca5a04 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:31.272344 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-01b273ae-3130-44fc-985a-6a426d65531e req-42fde18b-2688-4299-8494-4b62ebca5a04 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:31.272768 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-01b273ae-3130-44fc-985a-6a426d65531e req-42fde18b-2688-4299-8494-4b62ebca5a04 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:31.286814 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-01b273ae-3130-44fc-985a-6a426d65531e req-42fde18b-2688-4299-8494-4b62ebca5a04 service nova] Creating event network-vif-plugged:0af19f52-43e4-40ca-848a-d2902c3e8205 for instance aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8 on hv-cinder-maste Sep 04 01:06:31.292603 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-01b273ae-3130-44fc-985a-6a426d65531e req-42fde18b-2688-4299-8494-4b62ebca5a04 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036118 Sep 04 01:06:31.293340 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 106/243] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:06:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:06:31.982027 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-dc0398da-4d48-407e-952b-fda428adfa6a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:31.982555 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-dc0398da-4d48-407e-952b-fda428adfa6a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:31.989770 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dc0398da-4d48-407e-952b-fda428adfa6a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:31.990379 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dc0398da-4d48-407e-952b-fda428adfa6a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:31.990888 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dc0398da-4d48-407e-952b-fda428adfa6a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:31.991646 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dc0398da-4d48-407e-952b-fda428adfa6a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:31.992190 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dc0398da-4d48-407e-952b-fda428adfa6a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:31.992691 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dc0398da-4d48-407e-952b-fda428adfa6a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:32.189871 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-dc0398da-4d48-407e-952b-fda428adfa6a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:32.199203 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-dc0398da-4d48-407e-952b-fda428adfa6a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.220939 Sep 04 01:06:32.199769 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 138/244] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:31 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:06:32.208137 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-da25113d-cabd-4903-83a7-3fe2d4beaca8 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:32.208949 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-da25113d-cabd-4903-83a7-3fe2d4beaca8 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] [instance: aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:32.218845 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-da25113d-cabd-4903-83a7-3fe2d4beaca8 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:32.220212 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-da25113d-cabd-4903-83a7-3fe2d4beaca8 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:32.220641 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-da25113d-cabd-4903-83a7-3fe2d4beaca8 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:32.221732 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-da25113d-cabd-4903-83a7-3fe2d4beaca8 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:32.222361 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-da25113d-cabd-4903-83a7-3fe2d4beaca8 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:32.222852 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-da25113d-cabd-4903-83a7-3fe2d4beaca8 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:32.356098 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-da25113d-cabd-4903-83a7-3fe2d4beaca8 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af874cda15384e2f9160ccf6866296f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '528d5c15cc47445dbe46761067a1c54a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:32.364834 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-da25113d-cabd-4903-83a7-3fe2d4beaca8 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] 10.222.0.42 "GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8" status: 200 len: 1315 microversion: 2.1 time: 0.161873 Sep 04 01:06:32.365445 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 107/245] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:32 2022] GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8 => generated 1315 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:32.540280 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-01f98bca-0d66-4d6f-b933-1f1d849a9f86 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:32.540886 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-01f98bca-0d66-4d6f-b933-1f1d849a9f86 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:32.547718 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-01f98bca-0d66-4d6f-b933-1f1d849a9f86 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:32.548392 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-01f98bca-0d66-4d6f-b933-1f1d849a9f86 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:32.548821 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-01f98bca-0d66-4d6f-b933-1f1d849a9f86 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:32.550050 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-01f98bca-0d66-4d6f-b933-1f1d849a9f86 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:32.550754 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-01f98bca-0d66-4d6f-b933-1f1d849a9f86 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:32.551336 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-01f98bca-0d66-4d6f-b933-1f1d849a9f86 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:32.556805 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=100 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:32.566617 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.wsgi [None req-01f98bca-0d66-4d6f-b933-1f1d849a9f86 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] HTTP exception thrown: Instance 742cb4b8-f03c-4256-bb96-fc2cfff43734 could not be found. Sep 04 01:06:32.568299 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-01f98bca-0d66-4d6f-b933-1f1d849a9f86 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Request method failure captured: Sep 04 01:06:32.568299 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: request: DELETE /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 HTTP/1.1 Sep 04 01:06:32.568299 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept: application/json Sep 04 01:06:32.568299 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept-Encoding: identity Sep 04 01:06:32.568299 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Connection: close Sep 04 01:06:32.568299 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Length: 0 Sep 04 01:06:32.568299 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Type: application/json Sep 04 01:06:32.568299 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Host: 10.222.0.42 Sep 04 01:06:32.568299 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Openstack-System-Scope: None Sep 04 01:06:32.568299 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: User-Agent: python-urllib3/1.26.12 Sep 04 01:06:32.568299 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Auth-Token: gAAAAABjE_nDnngl7ftGavz-i2_6lnN2Oc2VvKaTiT-mXQQYKbwql02Y_PHxrmOsik8amb4rncT4NZZw0SrQ9gjmjUjdvAcquMu5ExeRYTuFG3mA6_Urv181U-EXuCZS1rfVDhwXABPfhIeZjcFWrOd62o-R4_0qHvBkErxliTR8plS61B0SPYs Sep 04 01:06:32.568299 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Id: None Sep 04 01:06:32.568299 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Name: None Sep 04 01:06:32.568299 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Identity-Status: Confirmed Sep 04 01:06:32.568299 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Is-Admin-Project: True Sep 04 01:06:32.568299 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Id: default Sep 04 01:06:32.568299 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Name: Default Sep 04 01:06:32.568299 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Id: e07d8b14fef34e04a0410f12e5dbd2fe Sep 04 01:06:32.568299 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Name: tempest-DeleteServersTestJSON-1716711868 Sep 04 01:06:32.568299 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Role: reader,member Sep 04 01:06:32.568299 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Roles: reader,member Sep 04 01:06:32.569721 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/e07d8b14fef34e04a0410f12e5dbd2fe"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_e07d8b14fef34e04a0410f12e5dbd2fe", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/e07d8b14fef34e04a0410f12e5dbd2fe"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/e07d8b14fef34e04a0410f12e5dbd2fe"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:06:32.569721 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant: tempest-DeleteServersTestJSON-1716711868 Sep 04 01:06:32.569721 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Id: e07d8b14fef34e04a0410f12e5dbd2fe Sep 04 01:06:32.569721 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Name: tempest-DeleteServersTestJSON-1716711868 Sep 04 01:06:32.569721 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User: tempest-DeleteServersTestJSON-1716711868-project Sep 04 01:06:32.569721 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Id: default Sep 04 01:06:32.569721 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Name: Default Sep 04 01:06:32.569721 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Id: b6b95b6814cf4cd8b77d87e7810c70e5 Sep 04 01:06:32.569721 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Name: tempest-DeleteServersTestJSON-1716711868-project Sep 04 01:06:32.569721 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: method: > Sep 04 01:06:32.569721 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: exception: Instance 742cb4b8-f03c-4256-bb96-fc2cfff43734 could not be found. Sep 04 01:06:32.569721 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: action_args: {'id': '742cb4b8-f03c-4256-bb96-fc2cfff43734'} Sep 04 01:06:32.570490 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:06:32.570490 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:06:32.570490 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:06:32.570490 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:06:32.570490 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:06:32.570490 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:06:32.570490 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:06:32.570490 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:06:32.570490 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:06:32.570490 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:06:32.570490 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 742cb4b8-f03c-4256-bb96-fc2cfff43734 could not be found. Sep 04 01:06:32.570490 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:06:32.570490 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:06:32.570490 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:06:32.570490 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:06:32.570490 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:06:32.570490 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:06:32.570490 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:06:32.570490 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:06:32.570490 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:06:32.571793 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:06:32.571793 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 04 01:06:32.571793 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 04 01:06:32.571793 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 04 01:06:32.571793 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 04 01:06:32.571793 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:06:32.571793 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:06:32.571793 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:06:32.571793 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:06:32.571793 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 742cb4b8-f03c-4256-bb96-fc2cfff43734 could not be found. Sep 04 01:06:32.571793 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:06:32.571793 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:06:32.571793 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:06:32.571793 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:06:32.571793 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:06:32.571793 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:06:32.571793 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:06:32.571793 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:06:32.571793 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 742cb4b8-f03c-4256-bb96-fc2cfff43734 could not be found. Sep 04 01:06:32.571793 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:06:32.573234 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-01f98bca-0d66-4d6f-b933-1f1d849a9f86 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Returning 404 to user: Instance 742cb4b8-f03c-4256-bb96-fc2cfff43734 could not be found. {{(pid=121036) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:06:32.573234 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-01f98bca-0d66-4d6f-b933-1f1d849a9f86 tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "DELETE /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 404 len: 111 microversion: 2.1 time: 0.035343 Sep 04 01:06:32.574978 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 139/246] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 4 01:06:32 2022] DELETE /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 04 01:06:32.578848 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-a76a54b6-98ba-4b61-95e3-c44ab5e4921b tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:32.582121 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-a76a54b6-98ba-4b61-95e3-c44ab5e4921b tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] [instance: 742cb4b8-f03c-4256-bb96-fc2cfff43734] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:32.586571 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a76a54b6-98ba-4b61-95e3-c44ab5e4921b tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:32.587068 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a76a54b6-98ba-4b61-95e3-c44ab5e4921b tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:32.587504 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a76a54b6-98ba-4b61-95e3-c44ab5e4921b tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:32.588202 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a76a54b6-98ba-4b61-95e3-c44ab5e4921b tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:32.588652 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a76a54b6-98ba-4b61-95e3-c44ab5e4921b tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:32.589092 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a76a54b6-98ba-4b61-95e3-c44ab5e4921b tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:32.603038 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.wsgi [None req-a76a54b6-98ba-4b61-95e3-c44ab5e4921b tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] HTTP exception thrown: Instance 742cb4b8-f03c-4256-bb96-fc2cfff43734 could not be found. Sep 04 01:06:32.605244 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-a76a54b6-98ba-4b61-95e3-c44ab5e4921b tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Request method failure captured: Sep 04 01:06:32.605244 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: request: GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 HTTP/1.1 Sep 04 01:06:32.605244 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept: application/json Sep 04 01:06:32.605244 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept-Encoding: identity Sep 04 01:06:32.605244 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Connection: close Sep 04 01:06:32.605244 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Length: 0 Sep 04 01:06:32.605244 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Type: application/json Sep 04 01:06:32.605244 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Host: 10.222.0.42 Sep 04 01:06:32.605244 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Openstack-System-Scope: None Sep 04 01:06:32.605244 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: User-Agent: python-urllib3/1.26.12 Sep 04 01:06:32.605244 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Auth-Token: gAAAAABjE_nDnngl7ftGavz-i2_6lnN2Oc2VvKaTiT-mXQQYKbwql02Y_PHxrmOsik8amb4rncT4NZZw0SrQ9gjmjUjdvAcquMu5ExeRYTuFG3mA6_Urv181U-EXuCZS1rfVDhwXABPfhIeZjcFWrOd62o-R4_0qHvBkErxliTR8plS61B0SPYs Sep 04 01:06:32.605244 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Id: None Sep 04 01:06:32.605244 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Name: None Sep 04 01:06:32.605244 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Identity-Status: Confirmed Sep 04 01:06:32.605244 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Is-Admin-Project: True Sep 04 01:06:32.605244 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Id: default Sep 04 01:06:32.605244 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Name: Default Sep 04 01:06:32.605244 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Id: e07d8b14fef34e04a0410f12e5dbd2fe Sep 04 01:06:32.605244 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Name: tempest-DeleteServersTestJSON-1716711868 Sep 04 01:06:32.605244 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Role: reader,member Sep 04 01:06:32.605244 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Roles: reader,member Sep 04 01:06:32.606760 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/e07d8b14fef34e04a0410f12e5dbd2fe"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_e07d8b14fef34e04a0410f12e5dbd2fe", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/e07d8b14fef34e04a0410f12e5dbd2fe"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/e07d8b14fef34e04a0410f12e5dbd2fe"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:06:32.606760 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant: tempest-DeleteServersTestJSON-1716711868 Sep 04 01:06:32.606760 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Id: e07d8b14fef34e04a0410f12e5dbd2fe Sep 04 01:06:32.606760 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Name: tempest-DeleteServersTestJSON-1716711868 Sep 04 01:06:32.606760 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User: tempest-DeleteServersTestJSON-1716711868-project Sep 04 01:06:32.606760 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Id: default Sep 04 01:06:32.606760 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Name: Default Sep 04 01:06:32.606760 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Id: b6b95b6814cf4cd8b77d87e7810c70e5 Sep 04 01:06:32.606760 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Name: tempest-DeleteServersTestJSON-1716711868-project Sep 04 01:06:32.606760 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: method: > Sep 04 01:06:32.606760 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: exception: Instance 742cb4b8-f03c-4256-bb96-fc2cfff43734 could not be found. Sep 04 01:06:32.606760 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: action_args: {'id': '742cb4b8-f03c-4256-bb96-fc2cfff43734'} Sep 04 01:06:32.607578 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:06:32.607578 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:06:32.607578 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:06:32.607578 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:06:32.607578 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:06:32.607578 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:06:32.607578 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:06:32.607578 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:06:32.607578 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:06:32.607578 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:06:32.607578 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 742cb4b8-f03c-4256-bb96-fc2cfff43734 could not be found. Sep 04 01:06:32.607578 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:06:32.607578 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:06:32.607578 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:06:32.607578 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:06:32.607578 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:06:32.607578 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:06:32.607578 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:06:32.607578 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:06:32.607578 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:06:32.609125 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:06:32.609125 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 04 01:06:32.609125 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 04 01:06:32.609125 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:06:32.609125 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:06:32.609125 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:06:32.609125 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:06:32.609125 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 742cb4b8-f03c-4256-bb96-fc2cfff43734 could not be found. Sep 04 01:06:32.609125 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:06:32.609125 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:06:32.609125 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:06:32.609125 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:06:32.609125 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:06:32.609125 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:06:32.609125 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:06:32.609125 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:06:32.609125 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 742cb4b8-f03c-4256-bb96-fc2cfff43734 could not be found. Sep 04 01:06:32.609125 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:06:32.613401 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-a76a54b6-98ba-4b61-95e3-c44ab5e4921b tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] Returning 404 to user: Instance 742cb4b8-f03c-4256-bb96-fc2cfff43734 could not be found. {{(pid=121037) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:06:32.613401 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-a76a54b6-98ba-4b61-95e3-c44ab5e4921b tempest-DeleteServersTestJSON-1716711868 tempest-DeleteServersTestJSON-1716711868-project] 10.222.0.42 "GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734" status: 404 len: 111 microversion: 2.1 time: 0.031547 Sep 04 01:06:32.613401 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 108/247] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:32 2022] GET /compute/v2.1/servers/742cb4b8-f03c-4256-bb96-fc2cfff43734 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 04 01:06:33.223467 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-6db4e206-2563-4724-ae92-368dc358694c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:33.224138 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-6db4e206-2563-4724-ae92-368dc358694c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:33.230765 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6db4e206-2563-4724-ae92-368dc358694c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:33.231250 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6db4e206-2563-4724-ae92-368dc358694c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:33.231681 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6db4e206-2563-4724-ae92-368dc358694c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:33.232366 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6db4e206-2563-4724-ae92-368dc358694c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:33.232844 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6db4e206-2563-4724-ae92-368dc358694c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:33.233494 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6db4e206-2563-4724-ae92-368dc358694c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:33.389552 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-3ad9e983-4a9f-4692-9104-ea5cf4e0c514 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:33.390830 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-3ad9e983-4a9f-4692-9104-ea5cf4e0c514 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] [instance: aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:33.399415 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3ad9e983-4a9f-4692-9104-ea5cf4e0c514 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:33.400854 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3ad9e983-4a9f-4692-9104-ea5cf4e0c514 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:33.400854 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3ad9e983-4a9f-4692-9104-ea5cf4e0c514 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:33.401846 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3ad9e983-4a9f-4692-9104-ea5cf4e0c514 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:33.402513 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3ad9e983-4a9f-4692-9104-ea5cf4e0c514 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:33.403304 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3ad9e983-4a9f-4692-9104-ea5cf4e0c514 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:33.590292 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-3ad9e983-4a9f-4692-9104-ea5cf4e0c514 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af874cda15384e2f9160ccf6866296f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '528d5c15cc47445dbe46761067a1c54a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:33.600329 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-3ad9e983-4a9f-4692-9104-ea5cf4e0c514 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] 10.222.0.42 "GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8" status: 200 len: 1315 microversion: 2.1 time: 0.215989 Sep 04 01:06:33.601330 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 109/248] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:33 2022] GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8 => generated 1315 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:33.697955 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-6db4e206-2563-4724-ae92-368dc358694c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:33.707426 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-6db4e206-2563-4724-ae92-368dc358694c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.487847 Sep 04 01:06:33.708042 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 140/249] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:33 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:34.624890 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-b8228309-0412-45a4-a4f7-e33b2b365bec tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:34.625485 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-b8228309-0412-45a4-a4f7-e33b2b365bec tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] [instance: aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:34.632550 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b8228309-0412-45a4-a4f7-e33b2b365bec tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:34.633518 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b8228309-0412-45a4-a4f7-e33b2b365bec tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:34.634127 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b8228309-0412-45a4-a4f7-e33b2b365bec tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:34.634853 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b8228309-0412-45a4-a4f7-e33b2b365bec tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:34.635487 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b8228309-0412-45a4-a4f7-e33b2b365bec tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:34.635912 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b8228309-0412-45a4-a4f7-e33b2b365bec tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:34.731650 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-c8f6e6b2-025e-48b6-99a2-bbc667580c32 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:34.732254 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-c8f6e6b2-025e-48b6-99a2-bbc667580c32 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:34.740254 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c8f6e6b2-025e-48b6-99a2-bbc667580c32 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:34.740982 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c8f6e6b2-025e-48b6-99a2-bbc667580c32 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:34.741532 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c8f6e6b2-025e-48b6-99a2-bbc667580c32 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:34.742219 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c8f6e6b2-025e-48b6-99a2-bbc667580c32 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:34.742822 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c8f6e6b2-025e-48b6-99a2-bbc667580c32 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:34.743244 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c8f6e6b2-025e-48b6-99a2-bbc667580c32 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:34.882532 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-b8228309-0412-45a4-a4f7-e33b2b365bec tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af874cda15384e2f9160ccf6866296f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '528d5c15cc47445dbe46761067a1c54a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:34.896271 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-b8228309-0412-45a4-a4f7-e33b2b365bec tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] 10.222.0.42 "GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8" status: 200 len: 1464 microversion: 2.1 time: 0.276249 Sep 04 01:06:34.897001 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 110/250] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:34 2022] GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8 => generated 1464 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:34.974856 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-32db45e6-3e25-4058-9bad-bf27141fc998 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:34.975471 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-32db45e6-3e25-4058-9bad-bf27141fc998 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] [instance: aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:34.982968 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-32db45e6-3e25-4058-9bad-bf27141fc998 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:34.989582 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-32db45e6-3e25-4058-9bad-bf27141fc998 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:34.989582 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-32db45e6-3e25-4058-9bad-bf27141fc998 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:34.989582 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-32db45e6-3e25-4058-9bad-bf27141fc998 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:34.989582 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-32db45e6-3e25-4058-9bad-bf27141fc998 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:34.989970 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-32db45e6-3e25-4058-9bad-bf27141fc998 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:35.023893 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-32db45e6-3e25-4058-9bad-bf27141fc998 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] [instance: aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8] Going to try to terminate instance {{(pid=121037) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 04 01:06:35.086523 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-c8f6e6b2-025e-48b6-99a2-bbc667580c32 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:35.120793 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-c8f6e6b2-025e-48b6-99a2-bbc667580c32 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.392668 Sep 04 01:06:35.132981 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 141/251] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:34 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:06:35.362097 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-32db45e6-3e25-4058-9bad-bf27141fc998 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] 10.222.0.42 "DELETE /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8" status: 204 len: 0 microversion: 2.1 time: 0.401884 Sep 04 01:06:35.363748 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 111/252] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 4 01:06:34 2022] DELETE /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 04 01:06:35.373745 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-138e1224-c717-4d4b-9eea-9df3a149c31b tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:35.374367 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-138e1224-c717-4d4b-9eea-9df3a149c31b tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] [instance: aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:35.381484 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-138e1224-c717-4d4b-9eea-9df3a149c31b tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:35.381971 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-138e1224-c717-4d4b-9eea-9df3a149c31b tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:35.382430 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-138e1224-c717-4d4b-9eea-9df3a149c31b tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:35.383182 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-138e1224-c717-4d4b-9eea-9df3a149c31b tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:35.383582 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-138e1224-c717-4d4b-9eea-9df3a149c31b tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:35.383982 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-138e1224-c717-4d4b-9eea-9df3a149c31b tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:35.622528 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-138e1224-c717-4d4b-9eea-9df3a149c31b tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af874cda15384e2f9160ccf6866296f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '528d5c15cc47445dbe46761067a1c54a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:35.642104 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-138e1224-c717-4d4b-9eea-9df3a149c31b tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] 10.222.0.42 "GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8" status: 200 len: 1470 microversion: 2.1 time: 0.270410 Sep 04 01:06:35.661245 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 142/253] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:35 2022] GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8 => generated 1470 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:36.145347 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-a9721462-db84-4ef3-b97d-1f1211724f6b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:36.146095 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-a9721462-db84-4ef3-b97d-1f1211724f6b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:36.160933 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a9721462-db84-4ef3-b97d-1f1211724f6b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:36.161471 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a9721462-db84-4ef3-b97d-1f1211724f6b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:36.161914 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a9721462-db84-4ef3-b97d-1f1211724f6b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:36.165889 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a9721462-db84-4ef3-b97d-1f1211724f6b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:36.165889 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a9721462-db84-4ef3-b97d-1f1211724f6b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:36.165889 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a9721462-db84-4ef3-b97d-1f1211724f6b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:36.375155 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-a9721462-db84-4ef3-b97d-1f1211724f6b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:36.385880 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-a9721462-db84-4ef3-b97d-1f1211724f6b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.245537 Sep 04 01:06:36.386666 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 112/254] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:36 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:06:36.670658 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-78d3e840-f995-4f8b-b947-4d495bd6dad4 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:36.671338 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-78d3e840-f995-4f8b-b947-4d495bd6dad4 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] [instance: aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:36.679025 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-78d3e840-f995-4f8b-b947-4d495bd6dad4 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:36.679732 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-78d3e840-f995-4f8b-b947-4d495bd6dad4 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:36.680708 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-78d3e840-f995-4f8b-b947-4d495bd6dad4 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:36.681789 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-78d3e840-f995-4f8b-b947-4d495bd6dad4 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:36.682387 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-78d3e840-f995-4f8b-b947-4d495bd6dad4 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:36.682971 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-78d3e840-f995-4f8b-b947-4d495bd6dad4 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:37.156000 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-78d3e840-f995-4f8b-b947-4d495bd6dad4 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af874cda15384e2f9160ccf6866296f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '528d5c15cc47445dbe46761067a1c54a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:37.164177 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-78d3e840-f995-4f8b-b947-4d495bd6dad4 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] 10.222.0.42 "GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8" status: 200 len: 1470 microversion: 2.1 time: 0.497614 Sep 04 01:06:37.177444 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 143/255] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:36 2022] GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8 => generated 1470 bytes in 513 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:37.410345 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-d36fc214-d41a-4591-aa0b-d929c2661d7c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:37.410959 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-d36fc214-d41a-4591-aa0b-d929c2661d7c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:37.417537 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d36fc214-d41a-4591-aa0b-d929c2661d7c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:37.418017 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d36fc214-d41a-4591-aa0b-d929c2661d7c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:37.418442 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d36fc214-d41a-4591-aa0b-d929c2661d7c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:37.419238 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d36fc214-d41a-4591-aa0b-d929c2661d7c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:37.419592 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d36fc214-d41a-4591-aa0b-d929c2661d7c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:37.420026 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d36fc214-d41a-4591-aa0b-d929c2661d7c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:37.422918 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:37.559118 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-22fd5642-45d8-4f12-894b-18c3560617f4 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-166532301", "imageRef": "a11315c3-ce81-478b-91e4-5353f189dfa7", "flavorRef": "452", "networks": [{"uuid": "f5f50aa9-3a0d-4b0d-aee8-3b2aef89717d"}]}} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:06:37.559118 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.network.neutron [None req-22fd5642-45d8-4f12-894b-18c3560617f4 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] validate_networks() for [('f5f50aa9-3a0d-4b0d-aee8-3b2aef89717d', None, None, None, None, None)] {{(pid=121036) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 04 01:06:37.660442 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-d36fc214-d41a-4591-aa0b-d929c2661d7c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:37.669329 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-d36fc214-d41a-4591-aa0b-d929c2661d7c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.263479 Sep 04 01:06:37.670145 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 113/256] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:37 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:38.084979 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-22fd5642-45d8-4f12-894b-18c3560617f4 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Flavor limits 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 04 01:06:38.086624 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-22fd5642-45d8-4f12-894b-18c3560617f4 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Image limits 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 04 01:06:38.087521 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-22fd5642-45d8-4f12-894b-18c3560617f4 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Flavor pref 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 04 01:06:38.088234 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-22fd5642-45d8-4f12-894b-18c3560617f4 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Image pref 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 04 01:06:38.089021 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-22fd5642-45d8-4f12-894b-18c3560617f4 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 04 01:06:38.099895 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-22fd5642-45d8-4f12-894b-18c3560617f4 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:38.105021 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-22fd5642-45d8-4f12-894b-18c3560617f4 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:38.105021 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-22fd5642-45d8-4f12-894b-18c3560617f4 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:38.105021 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-22fd5642-45d8-4f12-894b-18c3560617f4 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:38.105021 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-22fd5642-45d8-4f12-894b-18c3560617f4 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:38.105869 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-22fd5642-45d8-4f12-894b-18c3560617f4 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:38.160436 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-22fd5642-45d8-4f12-894b-18c3560617f4 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Getting quotas for project 6071c3a795ba4aa098e3af8cde306d58. Resources: {'ram', 'instances', 'cores'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 04 01:06:38.160436 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-22fd5642-45d8-4f12-894b-18c3560617f4 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Getting quotas for user 77de7bb68f3c45d680c8a78dc40df07b and project 6071c3a795ba4aa098e3af8cde306d58. Resources: {'ram', 'instances', 'cores'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 04 01:06:38.160436 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-22fd5642-45d8-4f12-894b-18c3560617f4 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Going to run 1 instances... {{(pid=121036) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 04 01:06:38.162004 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-22fd5642-45d8-4f12-894b-18c3560617f4 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: 2808d334-c39c-4a09-88c7-c1711cce4215] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a11315c3-ce81-478b-91e4-5353f189dfa7',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121036) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 04 01:06:38.178896 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-8eedbfa5-4d71-4ca1-ae88-7281721d4640 req-7ae8c0ff-a78d-403e-80fa-2cc476daca53 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8", "name": "network-vif-deleted", "tag": "0af19f52-43e4-40ca-848a-d2902c3e8205"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:06:38.204979 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-8eedbfa5-4d71-4ca1-ae88-7281721d4640 req-7ae8c0ff-a78d-403e-80fa-2cc476daca53 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:38.206969 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-8eedbfa5-4d71-4ca1-ae88-7281721d4640 req-7ae8c0ff-a78d-403e-80fa-2cc476daca53 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:38.207396 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-8eedbfa5-4d71-4ca1-ae88-7281721d4640 req-7ae8c0ff-a78d-403e-80fa-2cc476daca53 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:38.219277 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-8eedbfa5-4d71-4ca1-ae88-7281721d4640 req-7ae8c0ff-a78d-403e-80fa-2cc476daca53 service nova] Creating event network-vif-deleted:0af19f52-43e4-40ca-848a-d2902c3e8205 for instance aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8 on hv-cinder-maste Sep 04 01:06:38.225219 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-8eedbfa5-4d71-4ca1-ae88-7281721d4640 req-7ae8c0ff-a78d-403e-80fa-2cc476daca53 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.050057 Sep 04 01:06:38.226042 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 114/257] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:06:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:06:38.234274 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-618f90cb-77be-46ec-8773-1e3945c5c91a tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:38.239464 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-618f90cb-77be-46ec-8773-1e3945c5c91a tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] [instance: aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:38.243035 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-618f90cb-77be-46ec-8773-1e3945c5c91a tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:38.243516 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-618f90cb-77be-46ec-8773-1e3945c5c91a tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:38.243947 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-618f90cb-77be-46ec-8773-1e3945c5c91a tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:38.244660 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-618f90cb-77be-46ec-8773-1e3945c5c91a tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:38.245125 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-618f90cb-77be-46ec-8773-1e3945c5c91a tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:38.245549 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-618f90cb-77be-46ec-8773-1e3945c5c91a tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:38.337497 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-618f90cb-77be-46ec-8773-1e3945c5c91a tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af874cda15384e2f9160ccf6866296f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '528d5c15cc47445dbe46761067a1c54a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:38.341007 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-618f90cb-77be-46ec-8773-1e3945c5c91a tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] 10.222.0.42 "GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8" status: 200 len: 1428 microversion: 2.1 time: 0.111015 Sep 04 01:06:38.341795 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 115/258] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:38 2022] GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8 => generated 1428 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:38.370941 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-22fd5642-45d8-4f12-894b-18c3560617f4 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.888740 Sep 04 01:06:38.375389 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 144/259] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 4 01:06:37 2022] POST /compute/v2.1/servers => generated 380 bytes in 894 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 04 01:06:38.383007 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-7c542548-9d2f-4760-9d6b-f27c8da2f830 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:38.384139 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-7c542548-9d2f-4760-9d6b-f27c8da2f830 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: 2808d334-c39c-4a09-88c7-c1711cce4215] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:38.445060 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-7c542548-9d2f-4760-9d6b-f27c8da2f830 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:38.452062 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-7c542548-9d2f-4760-9d6b-f27c8da2f830 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215" status: 200 len: 1218 microversion: 2.1 time: 0.072861 Sep 04 01:06:38.452621 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 116/260] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:38 2022] GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215 => generated 1218 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:38.697643 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-6903f067-5657-4cf6-87cd-47999bf8c301 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:38.698286 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-6903f067-5657-4cf6-87cd-47999bf8c301 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:38.707126 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6903f067-5657-4cf6-87cd-47999bf8c301 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:38.707765 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6903f067-5657-4cf6-87cd-47999bf8c301 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:38.711307 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6903f067-5657-4cf6-87cd-47999bf8c301 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:38.711307 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6903f067-5657-4cf6-87cd-47999bf8c301 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:38.711307 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6903f067-5657-4cf6-87cd-47999bf8c301 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:38.711307 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6903f067-5657-4cf6-87cd-47999bf8c301 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:38.862288 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-6903f067-5657-4cf6-87cd-47999bf8c301 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:38.883700 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-6903f067-5657-4cf6-87cd-47999bf8c301 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.189867 Sep 04 01:06:38.884565 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 145/261] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:38 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:39.369388 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-17235cf6-8f2c-429a-9682-4face6a26080 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:39.369836 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-17235cf6-8f2c-429a-9682-4face6a26080 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] [instance: aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:39.379643 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-17235cf6-8f2c-429a-9682-4face6a26080 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:39.379643 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-17235cf6-8f2c-429a-9682-4face6a26080 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:39.379643 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-17235cf6-8f2c-429a-9682-4face6a26080 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:39.379643 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-17235cf6-8f2c-429a-9682-4face6a26080 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:39.379945 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-17235cf6-8f2c-429a-9682-4face6a26080 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:39.380120 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-17235cf6-8f2c-429a-9682-4face6a26080 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:39.412507 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-17235cf6-8f2c-429a-9682-4face6a26080 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:39.413064 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-17235cf6-8f2c-429a-9682-4face6a26080 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:39.413470 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-17235cf6-8f2c-429a-9682-4face6a26080 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:39.414065 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.objects.instance [None req-17235cf6-8f2c-429a-9682-4face6a26080 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lazy-loading 'fault' on Instance uuid aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8 {{(pid=121037) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:06:39.455829 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-17235cf6-8f2c-429a-9682-4face6a26080 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'af874cda15384e2f9160ccf6866296f6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '528d5c15cc47445dbe46761067a1c54a', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:39.463883 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-17235cf6-8f2c-429a-9682-4face6a26080 tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] 10.222.0.42 "GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8" status: 200 len: 1305 microversion: 2.1 time: 0.098807 Sep 04 01:06:39.464550 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 117/262] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:39 2022] GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8 => generated 1305 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:06:39.475704 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-14b70b39-4336-49b1-a707-a9327ed5c435 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:39.476320 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-14b70b39-4336-49b1-a707-a9327ed5c435 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: 2808d334-c39c-4a09-88c7-c1711cce4215] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:39.534232 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-14b70b39-4336-49b1-a707-a9327ed5c435 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:39.541724 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-14b70b39-4336-49b1-a707-a9327ed5c435 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215" status: 200 len: 1218 microversion: 2.1 time: 0.069790 Sep 04 01:06:39.542578 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 146/263] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:39 2022] GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215 => generated 1218 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:39.906905 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-6739acd2-cadd-40ed-bed6-118ccaa0718b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:39.907603 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-6739acd2-cadd-40ed-bed6-118ccaa0718b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:39.914387 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6739acd2-cadd-40ed-bed6-118ccaa0718b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:39.917193 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6739acd2-cadd-40ed-bed6-118ccaa0718b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:39.917193 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6739acd2-cadd-40ed-bed6-118ccaa0718b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:39.917193 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6739acd2-cadd-40ed-bed6-118ccaa0718b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:39.917193 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6739acd2-cadd-40ed-bed6-118ccaa0718b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:39.917744 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6739acd2-cadd-40ed-bed6-118ccaa0718b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:40.067347 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-6739acd2-cadd-40ed-bed6-118ccaa0718b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:40.075151 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-6739acd2-cadd-40ed-bed6-118ccaa0718b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.172091 Sep 04 01:06:40.075718 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 118/264] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:39 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:40.108150 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-3c8fd143-81a9-4ee2-bc09-75d19b045fe5 tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] Action: 'create', calling method: .version_select at 0x7fc10c861670>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-1060560188"}} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:06:40.494180 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-d70086b8-ee24-42f9-861e-b269b305e31f tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:40.494802 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-d70086b8-ee24-42f9-861e-b269b305e31f tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] [instance: aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:40.506612 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d70086b8-ee24-42f9-861e-b269b305e31f tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:40.507007 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d70086b8-ee24-42f9-861e-b269b305e31f tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:40.507437 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d70086b8-ee24-42f9-861e-b269b305e31f tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:40.508312 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d70086b8-ee24-42f9-861e-b269b305e31f tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:40.508757 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d70086b8-ee24-42f9-861e-b269b305e31f tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:40.509200 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d70086b8-ee24-42f9-861e-b269b305e31f tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:40.525356 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.wsgi [None req-d70086b8-ee24-42f9-861e-b269b305e31f tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] HTTP exception thrown: Instance aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8 could not be found. Sep 04 01:06:40.526984 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-d70086b8-ee24-42f9-861e-b269b305e31f tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Request method failure captured: Sep 04 01:06:40.526984 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: request: GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8 HTTP/1.1 Sep 04 01:06:40.526984 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept: application/json Sep 04 01:06:40.526984 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept-Encoding: identity Sep 04 01:06:40.526984 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Connection: close Sep 04 01:06:40.526984 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Length: 0 Sep 04 01:06:40.526984 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Type: application/json Sep 04 01:06:40.526984 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Host: 10.222.0.42 Sep 04 01:06:40.526984 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Openstack-System-Scope: None Sep 04 01:06:40.526984 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: User-Agent: python-urllib3/1.26.12 Sep 04 01:06:40.526984 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Auth-Token: gAAAAABjE_oEz-tCKLx76VBh3KUxjl3jXYKLe5ne231N7Tg84VT3nQLRk5sU4Xo8uYQJormuDTacHDixaZdicV3_k5m1z_-SrXJ9kHsJHJ3XXHtRiy4r2YLqoTrGIMn0Y4_00Q7BsZvhGuvLo2bai2PXgkn9QzD_lFh0uUR4hOBeA6CbYh3J6u0 Sep 04 01:06:40.526984 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Id: None Sep 04 01:06:40.526984 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Name: None Sep 04 01:06:40.526984 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Identity-Status: Confirmed Sep 04 01:06:40.526984 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Is-Admin-Project: True Sep 04 01:06:40.526984 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Id: default Sep 04 01:06:40.526984 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Name: Default Sep 04 01:06:40.526984 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Id: 528d5c15cc47445dbe46761067a1c54a Sep 04 01:06:40.526984 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Name: tempest-VolumesNegativeTest-1515737641 Sep 04 01:06:40.526984 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Role: reader,member Sep 04 01:06:40.526984 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Roles: reader,member Sep 04 01:06:40.528449 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/528d5c15cc47445dbe46761067a1c54a"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_528d5c15cc47445dbe46761067a1c54a", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/528d5c15cc47445dbe46761067a1c54a"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/528d5c15cc47445dbe46761067a1c54a"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:06:40.528449 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant: tempest-VolumesNegativeTest-1515737641 Sep 04 01:06:40.528449 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Id: 528d5c15cc47445dbe46761067a1c54a Sep 04 01:06:40.528449 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Name: tempest-VolumesNegativeTest-1515737641 Sep 04 01:06:40.528449 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User: tempest-VolumesNegativeTest-1515737641-project Sep 04 01:06:40.528449 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Id: default Sep 04 01:06:40.528449 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Name: Default Sep 04 01:06:40.528449 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Id: af874cda15384e2f9160ccf6866296f6 Sep 04 01:06:40.528449 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Name: tempest-VolumesNegativeTest-1515737641-project Sep 04 01:06:40.528449 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: method: > Sep 04 01:06:40.528449 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: exception: Instance aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8 could not be found. Sep 04 01:06:40.528449 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: action_args: {'id': 'aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8'} Sep 04 01:06:40.533417 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:06:40.533417 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:06:40.533417 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:06:40.533417 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:06:40.533417 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:06:40.533417 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:06:40.533417 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:06:40.533417 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:06:40.533417 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:06:40.533417 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:06:40.533417 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8 could not be found. Sep 04 01:06:40.533417 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:06:40.533417 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:06:40.533417 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:06:40.533417 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:06:40.533417 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:06:40.533417 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:06:40.533417 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:06:40.533417 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:06:40.533417 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:06:40.535787 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:06:40.535787 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 04 01:06:40.535787 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 04 01:06:40.535787 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:06:40.535787 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:06:40.535787 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:06:40.535787 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:06:40.535787 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8 could not be found. Sep 04 01:06:40.535787 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:06:40.535787 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:06:40.535787 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:06:40.535787 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:06:40.535787 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:06:40.535787 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:06:40.535787 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:06:40.535787 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:06:40.535787 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8 could not be found. Sep 04 01:06:40.535787 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:06:40.537042 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-d70086b8-ee24-42f9-861e-b269b305e31f tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] Returning 404 to user: Instance aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8 could not be found. {{(pid=121037) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:06:40.537042 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-d70086b8-ee24-42f9-861e-b269b305e31f tempest-VolumesNegativeTest-1515737641 tempest-VolumesNegativeTest-1515737641-project] 10.222.0.42 "GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8" status: 404 len: 111 microversion: 2.1 time: 0.038112 Sep 04 01:06:40.537042 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 119/265] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:40 2022] GET /compute/v2.1/servers/aea3d71b-c26e-4e64-a3d4-e2ae5cfa25d8 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 04 01:06:40.566680 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-7188ac8e-cb52-4f09-9520-178a880d2e5a tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:40.567312 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-7188ac8e-cb52-4f09-9520-178a880d2e5a tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: 2808d334-c39c-4a09-88c7-c1711cce4215] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:40.574040 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7188ac8e-cb52-4f09-9520-178a880d2e5a tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:40.574508 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7188ac8e-cb52-4f09-9520-178a880d2e5a tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:40.579992 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7188ac8e-cb52-4f09-9520-178a880d2e5a tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:40.579992 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7188ac8e-cb52-4f09-9520-178a880d2e5a tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:40.579992 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7188ac8e-cb52-4f09-9520-178a880d2e5a tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:40.579992 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7188ac8e-cb52-4f09-9520-178a880d2e5a tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:40.646689 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-7188ac8e-cb52-4f09-9520-178a880d2e5a tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:40.655661 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-7188ac8e-cb52-4f09-9520-178a880d2e5a tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215" status: 200 len: 1270 microversion: 2.1 time: 0.093582 Sep 04 01:06:40.656071 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 120/266] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:40 2022] GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215 => generated 1270 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:06:40.693797 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-3c8fd143-81a9-4ee2-bc09-75d19b045fe5 tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] 10.222.0.42 "POST /compute/v2.1/os-volumes" status: 200 len: 345 microversion: 2.1 time: 0.713891 Sep 04 01:06:40.695360 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 147/267] 10.222.0.42 () {60 vars in 1155 bytes} [Sun Sep 4 01:06:39 2022] POST /compute/v2.1/os-volumes => generated 345 bytes in 716 msecs (HTTP/1.1 200) 10 headers in 448 bytes (2 switches on core 0) Sep 04 01:06:40.709255 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-c4cb35de-08b2-4f37-96a4-15f4d24c3038 tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] Calling method '.version_select at 0x7fc10c6c6c10>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:40.761593 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-c4cb35de-08b2-4f37-96a4-15f4d24c3038 tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/64509ef0-13e6-43d5-a66c-a6cf6bd4d228" status: 200 len: 345 microversion: 2.1 time: 0.056571 Sep 04 01:06:40.769140 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 121/268] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Sep 4 01:06:40 2022] GET /compute/v2.1/os-volumes/64509ef0-13e6-43d5-a66c-a6cf6bd4d228 => generated 345 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:06:41.098578 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-5457a977-e80f-4215-a2d6-8372be0391b0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:41.099257 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-5457a977-e80f-4215-a2d6-8372be0391b0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:41.106003 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5457a977-e80f-4215-a2d6-8372be0391b0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:41.106487 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5457a977-e80f-4215-a2d6-8372be0391b0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:41.106923 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5457a977-e80f-4215-a2d6-8372be0391b0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:41.108452 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5457a977-e80f-4215-a2d6-8372be0391b0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:41.109024 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5457a977-e80f-4215-a2d6-8372be0391b0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:41.109530 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5457a977-e80f-4215-a2d6-8372be0391b0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:41.245270 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-5457a977-e80f-4215-a2d6-8372be0391b0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:41.256147 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-5457a977-e80f-4215-a2d6-8372be0391b0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.161439 Sep 04 01:06:41.256745 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 148/269] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:41 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:41.681557 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-3ee208c4-e1e6-4d4a-a752-1acb765fc1f4 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:41.682454 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-3ee208c4-e1e6-4d4a-a752-1acb765fc1f4 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: 2808d334-c39c-4a09-88c7-c1711cce4215] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:41.699032 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3ee208c4-e1e6-4d4a-a752-1acb765fc1f4 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:41.699032 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3ee208c4-e1e6-4d4a-a752-1acb765fc1f4 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:41.699032 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3ee208c4-e1e6-4d4a-a752-1acb765fc1f4 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:41.699032 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3ee208c4-e1e6-4d4a-a752-1acb765fc1f4 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:41.700721 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3ee208c4-e1e6-4d4a-a752-1acb765fc1f4 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:41.700721 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3ee208c4-e1e6-4d4a-a752-1acb765fc1f4 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:41.776461 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-5e2fbf79-e567-42a7-9af6-1ea3e1fbafee tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] Calling method '.version_select at 0x7fc10c774b80>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:41.812459 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-3ee208c4-e1e6-4d4a-a752-1acb765fc1f4 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:41.825758 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-3ee208c4-e1e6-4d4a-a752-1acb765fc1f4 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215" status: 200 len: 1276 microversion: 2.1 time: 0.148410 Sep 04 01:06:41.826636 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 122/270] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:41 2022] GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215 => generated 1276 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:41.831096 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-5e2fbf79-e567-42a7-9af6-1ea3e1fbafee tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/64509ef0-13e6-43d5-a66c-a6cf6bd4d228" status: 200 len: 346 microversion: 2.1 time: 0.059109 Sep 04 01:06:41.831801 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 149/271] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Sep 4 01:06:41 2022] GET /compute/v2.1/os-volumes/64509ef0-13e6-43d5-a66c-a6cf6bd4d228 => generated 346 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:06:41.857154 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-12134dbf-eef2-44ac-b4ff-806f92325ebd tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] Action: 'create', calling method: .version_select at 0x7fc10c6c64c0>, body: {"snapshot": {"volume_id": "64509ef0-13e6-43d5-a66c-a6cf6bd4d228", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-1257532887"}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:06:42.279820 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b510b80d-d5d1-4fdc-8e98-abc1ba3ad5c0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:42.280467 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-b510b80d-d5d1-4fdc-8e98-abc1ba3ad5c0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:42.287563 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b510b80d-d5d1-4fdc-8e98-abc1ba3ad5c0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:42.287751 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b510b80d-d5d1-4fdc-8e98-abc1ba3ad5c0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:42.288075 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b510b80d-d5d1-4fdc-8e98-abc1ba3ad5c0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:42.288817 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b510b80d-d5d1-4fdc-8e98-abc1ba3ad5c0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:42.289294 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b510b80d-d5d1-4fdc-8e98-abc1ba3ad5c0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:42.289888 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b510b80d-d5d1-4fdc-8e98-abc1ba3ad5c0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:42.412420 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-12134dbf-eef2-44ac-b4ff-806f92325ebd tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] 10.222.0.42 "POST /compute/v2.1/os-snapshots" status: 200 len: 287 microversion: 2.1 time: 0.560191 Sep 04 01:06:42.419105 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 123/272] 10.222.0.42 () {60 vars in 1162 bytes} [Sun Sep 4 01:06:41 2022] POST /compute/v2.1/os-snapshots => generated 287 bytes in 567 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:06:42.441149 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-7e405c97-5f4d-4238-87d4-f909cbec0381 tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] Calling method '.version_select at 0x7fc10c706e50>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:42.458842 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-b510b80d-d5d1-4fdc-8e98-abc1ba3ad5c0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:42.489346 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-b510b80d-d5d1-4fdc-8e98-abc1ba3ad5c0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.212697 Sep 04 01:06:42.493202 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 150/273] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:42 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:42.508373 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-7e405c97-5f4d-4238-87d4-f909cbec0381 tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/f80519bc-89f3-43a3-8be8-484e8a772e63" status: 200 len: 287 microversion: 2.1 time: 0.088135 Sep 04 01:06:42.509770 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 124/274] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Sep 4 01:06:42 2022] GET /compute/v2.1/os-snapshots/f80519bc-89f3-43a3-8be8-484e8a772e63 => generated 287 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:06:42.523199 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-e3d6cab0-f093-4544-add3-0f4baf097d51 tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] Calling method '.version_select at 0x7fc10c741f70>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:42.566423 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-e3d6cab0-f093-4544-add3-0f4baf097d51 tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots" status: 200 len: 290 microversion: 2.1 time: 0.046842 Sep 04 01:06:42.566984 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 151/275] 10.222.0.42 () {58 vars in 1140 bytes} [Sun Sep 4 01:06:42 2022] GET /compute/v2.1/os-snapshots => generated 290 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:06:42.581069 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-8ee7ad31-0c3a-4e49-86d6-5d69a842117b tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] Calling method '.version_select at 0x7fc10c737820>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:42.621176 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-8ee7ad31-0c3a-4e49-86d6-5d69a842117b tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/f80519bc-89f3-43a3-8be8-484e8a772e63" status: 200 len: 287 microversion: 2.1 time: 0.044644 Sep 04 01:06:42.621847 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 125/276] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Sep 4 01:06:42 2022] GET /compute/v2.1/os-snapshots/f80519bc-89f3-43a3-8be8-484e8a772e63 => generated 287 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:06:42.853601 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-ed86a7a4-e47e-485f-935c-2cec2edfefdd tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:42.854839 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-ed86a7a4-e47e-485f-935c-2cec2edfefdd tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: 2808d334-c39c-4a09-88c7-c1711cce4215] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:42.868167 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ed86a7a4-e47e-485f-935c-2cec2edfefdd tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:42.868167 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ed86a7a4-e47e-485f-935c-2cec2edfefdd tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:42.868167 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ed86a7a4-e47e-485f-935c-2cec2edfefdd tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:42.868167 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ed86a7a4-e47e-485f-935c-2cec2edfefdd tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:42.868760 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ed86a7a4-e47e-485f-935c-2cec2edfefdd tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:42.868760 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ed86a7a4-e47e-485f-935c-2cec2edfefdd tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:43.012160 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ed86a7a4-e47e-485f-935c-2cec2edfefdd tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:43.020807 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-ed86a7a4-e47e-485f-935c-2cec2edfefdd tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215" status: 200 len: 1318 microversion: 2.1 time: 0.171711 Sep 04 01:06:43.021521 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 152/277] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:42 2022] GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215 => generated 1318 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:06:43.085939 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-0bf6916c-19d2-418f-9f09-439462bdeeb6 req-6c399b28-b209-400c-95b2-401ea91c5ec1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2808d334-c39c-4a09-88c7-c1711cce4215", "tag": "449add0b-213e-49af-b0f2-5656f19d796f"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:06:43.096018 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-0bf6916c-19d2-418f-9f09-439462bdeeb6 req-6c399b28-b209-400c-95b2-401ea91c5ec1 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:43.096947 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-0bf6916c-19d2-418f-9f09-439462bdeeb6 req-6c399b28-b209-400c-95b2-401ea91c5ec1 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:43.097399 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-0bf6916c-19d2-418f-9f09-439462bdeeb6 req-6c399b28-b209-400c-95b2-401ea91c5ec1 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:43.251440 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-0bf6916c-19d2-418f-9f09-439462bdeeb6 req-6c399b28-b209-400c-95b2-401ea91c5ec1 service nova] Creating event network-changed:449add0b-213e-49af-b0f2-5656f19d796f for instance 2808d334-c39c-4a09-88c7-c1711cce4215 on hv-cinder-maste Sep 04 01:06:43.256722 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-0bf6916c-19d2-418f-9f09-439462bdeeb6 req-6c399b28-b209-400c-95b2-401ea91c5ec1 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.175026 Sep 04 01:06:43.257361 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 126/278] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:06:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:06:43.514533 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b5c9b24c-9ae4-4d05-ab68-62fa087a4d80 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:43.515135 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-b5c9b24c-9ae4-4d05-ab68-62fa087a4d80 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:43.521507 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b5c9b24c-9ae4-4d05-ab68-62fa087a4d80 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:43.521811 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b5c9b24c-9ae4-4d05-ab68-62fa087a4d80 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:43.522242 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b5c9b24c-9ae4-4d05-ab68-62fa087a4d80 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:43.522931 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b5c9b24c-9ae4-4d05-ab68-62fa087a4d80 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:43.523379 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b5c9b24c-9ae4-4d05-ab68-62fa087a4d80 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:43.523870 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b5c9b24c-9ae4-4d05-ab68-62fa087a4d80 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:43.634653 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-dc34eb09-4f2a-44b6-806a-50373c28ec00 tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] Calling method '.version_select at 0x7fc10c6c63a0>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:43.669962 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-dc34eb09-4f2a-44b6-806a-50373c28ec00 tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/f80519bc-89f3-43a3-8be8-484e8a772e63" status: 200 len: 287 microversion: 2.1 time: 0.039392 Sep 04 01:06:43.670576 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 127/279] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Sep 4 01:06:43 2022] GET /compute/v2.1/os-snapshots/f80519bc-89f3-43a3-8be8-484e8a772e63 => generated 287 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:06:43.687222 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-b5c9b24c-9ae4-4d05-ab68-62fa087a4d80 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:43.695760 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-b5c9b24c-9ae4-4d05-ab68-62fa087a4d80 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.185418 Sep 04 01:06:43.696319 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 153/280] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:43 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:44.044362 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-45ca017e-bd34-4b01-ae50-b25e1a5baa23 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:44.045117 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-45ca017e-bd34-4b01-ae50-b25e1a5baa23 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: 2808d334-c39c-4a09-88c7-c1711cce4215] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:44.054576 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-45ca017e-bd34-4b01-ae50-b25e1a5baa23 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:44.054576 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-45ca017e-bd34-4b01-ae50-b25e1a5baa23 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:44.054576 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-45ca017e-bd34-4b01-ae50-b25e1a5baa23 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:44.054576 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-45ca017e-bd34-4b01-ae50-b25e1a5baa23 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:44.054924 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-45ca017e-bd34-4b01-ae50-b25e1a5baa23 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:44.055039 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-45ca017e-bd34-4b01-ae50-b25e1a5baa23 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:44.203945 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-45ca017e-bd34-4b01-ae50-b25e1a5baa23 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:44.212510 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-45ca017e-bd34-4b01-ae50-b25e1a5baa23 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215" status: 200 len: 1318 microversion: 2.1 time: 0.172498 Sep 04 01:06:44.221328 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 128/281] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:44 2022] GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215 => generated 1318 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:44.681991 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-ddf6ec51-bf2a-48e6-93fa-cf5f05650670 tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] Calling method '.version_select at 0x7fc10c76d160>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:44.722693 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-ddf6ec51-bf2a-48e6-93fa-cf5f05650670 tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/f80519bc-89f3-43a3-8be8-484e8a772e63" status: 200 len: 287 microversion: 2.1 time: 0.040673 Sep 04 01:06:44.722785 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-14432700-93a3-4c46-9d0a-0289522921f4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:44.722885 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 154/282] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Sep 4 01:06:44 2022] GET /compute/v2.1/os-snapshots/f80519bc-89f3-43a3-8be8-484e8a772e63 => generated 287 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:06:44.722960 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-14432700-93a3-4c46-9d0a-0289522921f4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:44.727801 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-14432700-93a3-4c46-9d0a-0289522921f4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:44.728286 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-14432700-93a3-4c46-9d0a-0289522921f4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:44.728732 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-14432700-93a3-4c46-9d0a-0289522921f4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:44.729467 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-14432700-93a3-4c46-9d0a-0289522921f4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:44.729918 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-14432700-93a3-4c46-9d0a-0289522921f4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:44.730341 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-14432700-93a3-4c46-9d0a-0289522921f4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:44.878681 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-14432700-93a3-4c46-9d0a-0289522921f4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:44.886353 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-14432700-93a3-4c46-9d0a-0289522921f4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.171098 Sep 04 01:06:44.886925 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 129/283] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:44 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:45.236222 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-ecf2fd0f-4fa2-4ff1-ab92-be13b995d7ca tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:45.236910 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-ecf2fd0f-4fa2-4ff1-ab92-be13b995d7ca tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: 2808d334-c39c-4a09-88c7-c1711cce4215] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:45.243408 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ecf2fd0f-4fa2-4ff1-ab92-be13b995d7ca tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:45.243893 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ecf2fd0f-4fa2-4ff1-ab92-be13b995d7ca tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:45.248313 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ecf2fd0f-4fa2-4ff1-ab92-be13b995d7ca tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:45.248313 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ecf2fd0f-4fa2-4ff1-ab92-be13b995d7ca tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:45.248313 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ecf2fd0f-4fa2-4ff1-ab92-be13b995d7ca tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:45.248313 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ecf2fd0f-4fa2-4ff1-ab92-be13b995d7ca tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:45.377129 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ecf2fd0f-4fa2-4ff1-ab92-be13b995d7ca tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:45.385203 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-ecf2fd0f-4fa2-4ff1-ab92-be13b995d7ca tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215" status: 200 len: 1318 microversion: 2.1 time: 0.153422 Sep 04 01:06:45.385920 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 155/284] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:45 2022] GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215 => generated 1318 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:45.732962 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-edfc8eb7-1958-492c-ae13-abadb40f13e3 tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] Calling method '.version_select at 0x7fc10c84caf0>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:45.767578 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-edfc8eb7-1958-492c-ae13-abadb40f13e3 tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/f80519bc-89f3-43a3-8be8-484e8a772e63" status: 200 len: 287 microversion: 2.1 time: 0.038384 Sep 04 01:06:45.768783 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 130/285] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Sep 4 01:06:45 2022] GET /compute/v2.1/os-snapshots/f80519bc-89f3-43a3-8be8-484e8a772e63 => generated 287 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:06:45.909304 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b3c7fdb3-4904-49c8-bf72-1fbf5b9cf57c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:45.909929 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-b3c7fdb3-4904-49c8-bf72-1fbf5b9cf57c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:45.915826 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b3c7fdb3-4904-49c8-bf72-1fbf5b9cf57c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:45.916304 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b3c7fdb3-4904-49c8-bf72-1fbf5b9cf57c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:45.916787 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b3c7fdb3-4904-49c8-bf72-1fbf5b9cf57c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:45.917515 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b3c7fdb3-4904-49c8-bf72-1fbf5b9cf57c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:45.921594 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b3c7fdb3-4904-49c8-bf72-1fbf5b9cf57c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:45.921594 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b3c7fdb3-4904-49c8-bf72-1fbf5b9cf57c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:46.048733 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-b3c7fdb3-4904-49c8-bf72-1fbf5b9cf57c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:46.058236 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-b3c7fdb3-4904-49c8-bf72-1fbf5b9cf57c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.153075 Sep 04 01:06:46.058795 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 156/286] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:45 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:46.407450 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-8154017e-a50b-4906-b3f8-63d6b3e81349 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:46.408058 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-8154017e-a50b-4906-b3f8-63d6b3e81349 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: 2808d334-c39c-4a09-88c7-c1711cce4215] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:46.414308 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8154017e-a50b-4906-b3f8-63d6b3e81349 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:46.424862 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8154017e-a50b-4906-b3f8-63d6b3e81349 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:46.424862 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8154017e-a50b-4906-b3f8-63d6b3e81349 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:46.424862 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8154017e-a50b-4906-b3f8-63d6b3e81349 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:46.424862 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8154017e-a50b-4906-b3f8-63d6b3e81349 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:46.425253 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8154017e-a50b-4906-b3f8-63d6b3e81349 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:46.547010 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-8154017e-a50b-4906-b3f8-63d6b3e81349 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:46.556227 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-8154017e-a50b-4906-b3f8-63d6b3e81349 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215" status: 200 len: 1318 microversion: 2.1 time: 0.152850 Sep 04 01:06:46.556783 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 131/287] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:46 2022] GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215 => generated 1318 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:46.781045 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-f398fd1d-5e48-4014-b0d9-6c304ab88913 tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] Calling method '.version_select at 0x7fc10c774310>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:46.810791 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-f398fd1d-5e48-4014-b0d9-6c304ab88913 tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/f80519bc-89f3-43a3-8be8-484e8a772e63" status: 200 len: 288 microversion: 2.1 time: 0.033529 Sep 04 01:06:46.811347 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 157/288] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Sep 4 01:06:46 2022] GET /compute/v2.1/os-snapshots/f80519bc-89f3-43a3-8be8-484e8a772e63 => generated 288 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:06:46.823359 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-6a2da17f-7f22-4544-8c79-34e8b0cb502e tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] Calling method '.version_select at 0x7fc10c875430>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:46.966020 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-6a2da17f-7f22-4544-8c79-34e8b0cb502e tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] 10.222.0.42 "DELETE /compute/v2.1/os-snapshots/f80519bc-89f3-43a3-8be8-484e8a772e63" status: 202 len: 0 microversion: 2.1 time: 0.146361 Sep 04 01:06:46.966599 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 132/289] 10.222.0.42 () {58 vars in 1254 bytes} [Sun Sep 4 01:06:46 2022] DELETE /compute/v2.1/os-snapshots/f80519bc-89f3-43a3-8be8-484e8a772e63 => generated 0 bytes in 148 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 04 01:06:46.974822 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-603c1b42-3f99-4151-abc1-b4cf87cba5d6 tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] Calling method '.version_select at 0x7fc10d41ef70>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:47.008342 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-603c1b42-3f99-4151-abc1-b4cf87cba5d6 tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/f80519bc-89f3-43a3-8be8-484e8a772e63" status: 200 len: 287 microversion: 2.1 time: 0.037318 Sep 04 01:06:47.009138 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 158/290] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Sep 4 01:06:46 2022] GET /compute/v2.1/os-snapshots/f80519bc-89f3-43a3-8be8-484e8a772e63 => generated 287 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:06:47.082590 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-598bf4b4-72b9-4f15-b6c6-9021db5c2dc4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:47.083724 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-598bf4b4-72b9-4f15-b6c6-9021db5c2dc4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:47.090271 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-598bf4b4-72b9-4f15-b6c6-9021db5c2dc4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:47.090849 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-598bf4b4-72b9-4f15-b6c6-9021db5c2dc4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:47.091353 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-598bf4b4-72b9-4f15-b6c6-9021db5c2dc4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:47.092150 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-598bf4b4-72b9-4f15-b6c6-9021db5c2dc4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:47.092671 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-598bf4b4-72b9-4f15-b6c6-9021db5c2dc4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:47.093295 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-598bf4b4-72b9-4f15-b6c6-9021db5c2dc4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:47.386232 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-598bf4b4-72b9-4f15-b6c6-9021db5c2dc4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:47.394447 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-598bf4b4-72b9-4f15-b6c6-9021db5c2dc4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.316052 Sep 04 01:06:47.395013 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 133/291] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:47 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:47.580116 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-32a1227d-7930-4c23-a668-a9bc46cba843 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:47.581005 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-32a1227d-7930-4c23-a668-a9bc46cba843 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: 2808d334-c39c-4a09-88c7-c1711cce4215] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:47.583141 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=6 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:47.588992 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-32a1227d-7930-4c23-a668-a9bc46cba843 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:47.589542 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-32a1227d-7930-4c23-a668-a9bc46cba843 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:47.589869 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-32a1227d-7930-4c23-a668-a9bc46cba843 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:47.590561 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-32a1227d-7930-4c23-a668-a9bc46cba843 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:47.591145 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-32a1227d-7930-4c23-a668-a9bc46cba843 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:47.591430 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-32a1227d-7930-4c23-a668-a9bc46cba843 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:47.730025 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-32a1227d-7930-4c23-a668-a9bc46cba843 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:47.738610 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-32a1227d-7930-4c23-a668-a9bc46cba843 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215" status: 200 len: 1318 microversion: 2.1 time: 0.162513 Sep 04 01:06:47.749356 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 159/292] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:47 2022] GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215 => generated 1318 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:06:48.020671 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-d7eb85f0-b873-4e1e-8e53-3341ad55c967 tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] Calling method '.version_select at 0x7fc10c84c5e0>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:48.046185 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.wsgi [None req-d7eb85f0-b873-4e1e-8e53-3341ad55c967 tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] HTTP exception thrown: Snapshot f80519bc-89f3-43a3-8be8-484e8a772e63 could not be found. Sep 04 01:06:48.049980 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-d7eb85f0-b873-4e1e-8e53-3341ad55c967 tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] Request method failure captured: Sep 04 01:06:48.049980 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: request: GET /compute/v2.1/os-snapshots/f80519bc-89f3-43a3-8be8-484e8a772e63 HTTP/1.1 Sep 04 01:06:48.049980 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept: application/json Sep 04 01:06:48.049980 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept-Encoding: identity Sep 04 01:06:48.049980 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Connection: close Sep 04 01:06:48.049980 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Length: 0 Sep 04 01:06:48.049980 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Type: application/json Sep 04 01:06:48.049980 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Host: 10.222.0.42 Sep 04 01:06:48.049980 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Openstack-System-Scope: None Sep 04 01:06:48.049980 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: User-Agent: python-urllib3/1.26.12 Sep 04 01:06:48.049980 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Auth-Token: gAAAAABjE_of3TI7KqM3ooPmOchCj0Ut2Wq36cGVH4FUfdga2xsxLorCG9ZucGeQpVSFwI9tAwUqkuVB1_SMMzJ2QQvCAmxhxK-BuRw4zwxmGU4nd1PeiuFadlv7GNJs6-RsXXfjZUjzvXgI7XhYicagqGeHE-78QAqrfBZWEB2F9CiNIgd5CVM Sep 04 01:06:48.049980 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Id: None Sep 04 01:06:48.049980 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Name: None Sep 04 01:06:48.049980 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Identity-Status: Confirmed Sep 04 01:06:48.049980 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Is-Admin-Project: True Sep 04 01:06:48.049980 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Id: default Sep 04 01:06:48.049980 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Name: Default Sep 04 01:06:48.049980 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Id: 07a7237c4f0c4f3e8e43aa14ec0a9064 Sep 04 01:06:48.049980 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1932271488 Sep 04 01:06:48.049980 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Role: reader,member Sep 04 01:06:48.049980 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Roles: reader,member Sep 04 01:06:48.051440 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/07a7237c4f0c4f3e8e43aa14ec0a9064"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_07a7237c4f0c4f3e8e43aa14ec0a9064", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/07a7237c4f0c4f3e8e43aa14ec0a9064"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/07a7237c4f0c4f3e8e43aa14ec0a9064"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:06:48.051440 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1932271488 Sep 04 01:06:48.051440 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Id: 07a7237c4f0c4f3e8e43aa14ec0a9064 Sep 04 01:06:48.051440 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1932271488 Sep 04 01:06:48.051440 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User: tempest-VolumesSnapshotsTestJSON-1932271488-project Sep 04 01:06:48.051440 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Id: default Sep 04 01:06:48.051440 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Name: Default Sep 04 01:06:48.051440 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Id: f55f74f18e4c4e77b2d9d8b68ae0ea90 Sep 04 01:06:48.051440 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1932271488-project Sep 04 01:06:48.051440 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: method: Sep 04 01:06:48.051440 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: exception: Snapshot f80519bc-89f3-43a3-8be8-484e8a772e63 could not be found. Sep 04 01:06:48.051440 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: action_args: {'id': 'f80519bc-89f3-43a3-8be8-484e8a772e63'} Sep 04 01:06:48.051440 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:06:48.052288 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:06:48.052288 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Sep 04 01:06:48.052288 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Sep 04 01:06:48.052288 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Sep 04 01:06:48.052288 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Sep 04 01:06:48.052288 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Sep 04 01:06:48.052288 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Sep 04 01:06:48.052288 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 04 01:06:48.052288 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 04 01:06:48.052288 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 04 01:06:48.052288 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 04 01:06:48.052288 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 04 01:06:48.052288 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 04 01:06:48.052288 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 04 01:06:48.052288 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 04 01:06:48.052288 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot f80519bc-89f3-43a3-8be8-484e8a772e63 could not be found. (HTTP 404) (Request-ID: req-863a395f-c994-4bbd-a75f-c3dcaac40bac) Sep 04 01:06:48.052288 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:06:48.052288 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:06:48.070762 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:06:48.070762 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:06:48.070762 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Sep 04 01:06:48.070762 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Sep 04 01:06:48.070762 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 04 01:06:48.070762 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 04 01:06:48.070762 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Sep 04 01:06:48.070762 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Sep 04 01:06:48.070762 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 04 01:06:48.070762 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 04 01:06:48.070762 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Sep 04 01:06:48.070762 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Sep 04 01:06:48.070762 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Sep 04 01:06:48.070762 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Sep 04 01:06:48.070762 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Sep 04 01:06:48.070762 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Sep 04 01:06:48.070762 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 04 01:06:48.070762 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 04 01:06:48.070762 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 04 01:06:48.072023 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 04 01:06:48.072023 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 04 01:06:48.072023 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 04 01:06:48.072023 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 04 01:06:48.072023 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 04 01:06:48.072023 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot f80519bc-89f3-43a3-8be8-484e8a772e63 could not be found. Sep 04 01:06:48.072023 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:06:48.072023 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:06:48.072023 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:06:48.072023 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:06:48.072023 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:06:48.072023 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:06:48.072023 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:06:48.072023 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:06:48.072023 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 04 01:06:48.072023 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 04 01:06:48.072023 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:06:48.072023 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:06:48.072023 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Sep 04 01:06:48.072023 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:06:48.073338 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot f80519bc-89f3-43a3-8be8-484e8a772e63 could not be found. Sep 04 01:06:48.073338 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:06:48.073338 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:06:48.073338 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:06:48.073338 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:06:48.073338 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:06:48.073338 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:06:48.073338 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:06:48.073338 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:06:48.073338 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot f80519bc-89f3-43a3-8be8-484e8a772e63 could not be found. Sep 04 01:06:48.073338 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:06:48.073338 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-d7eb85f0-b873-4e1e-8e53-3341ad55c967 tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] Returning 404 to user: Snapshot f80519bc-89f3-43a3-8be8-484e8a772e63 could not be found. {{(pid=121037) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:06:48.073338 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-d7eb85f0-b873-4e1e-8e53-3341ad55c967 tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] 10.222.0.42 "GET /compute/v2.1/os-snapshots/f80519bc-89f3-43a3-8be8-484e8a772e63" status: 404 len: 111 microversion: 2.1 time: 0.036623 Sep 04 01:06:48.074269 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 134/293] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Sep 4 01:06:48 2022] GET /compute/v2.1/os-snapshots/f80519bc-89f3-43a3-8be8-484e8a772e63 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 04 01:06:48.074353 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-a521f681-3c41-4458-afea-cfea043a4a21 tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] Calling method '.version_select at 0x7fc10c7641f0>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:48.113795 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell0:SELECT=2 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:48.245767 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-a521f681-3c41-4458-afea-cfea043a4a21 tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] 10.222.0.42 "DELETE /compute/v2.1/os-volumes/64509ef0-13e6-43d5-a66c-a6cf6bd4d228" status: 202 len: 0 microversion: 2.1 time: 0.185462 Sep 04 01:06:48.247562 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 160/294] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Sep 4 01:06:48 2022] DELETE /compute/v2.1/os-volumes/64509ef0-13e6-43d5-a66c-a6cf6bd4d228 => generated 0 bytes in 187 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 04 01:06:48.255002 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-16f88462-0dba-4465-899d-8858c4190699 tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] Calling method '.version_select at 0x7fc10c8828b0>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:48.305744 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-16f88462-0dba-4465-899d-8858c4190699 tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/64509ef0-13e6-43d5-a66c-a6cf6bd4d228" status: 200 len: 345 microversion: 2.1 time: 0.054922 Sep 04 01:06:48.306500 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 135/295] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Sep 4 01:06:48 2022] GET /compute/v2.1/os-volumes/64509ef0-13e6-43d5-a66c-a6cf6bd4d228 => generated 345 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:06:48.417874 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-d687f5b6-47ec-40f6-95ce-828a4d8a0b4b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:48.418514 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-d687f5b6-47ec-40f6-95ce-828a4d8a0b4b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:48.425047 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d687f5b6-47ec-40f6-95ce-828a4d8a0b4b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:48.425938 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d687f5b6-47ec-40f6-95ce-828a4d8a0b4b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:48.426524 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d687f5b6-47ec-40f6-95ce-828a4d8a0b4b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:48.431180 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d687f5b6-47ec-40f6-95ce-828a4d8a0b4b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:48.431180 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d687f5b6-47ec-40f6-95ce-828a4d8a0b4b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:48.431180 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d687f5b6-47ec-40f6-95ce-828a4d8a0b4b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:48.572115 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-d687f5b6-47ec-40f6-95ce-828a4d8a0b4b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:48.580211 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-d687f5b6-47ec-40f6-95ce-828a4d8a0b4b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1519 microversion: 2.1 time: 0.166446 Sep 04 01:06:48.580845 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 161/296] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:48 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1519 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:48.760124 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell0:SELECT=2 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:48.764429 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-1215c83a-df9c-4542-8788-953a01b63753 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:48.765043 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-1215c83a-df9c-4542-8788-953a01b63753 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: 2808d334-c39c-4a09-88c7-c1711cce4215] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:48.771838 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1215c83a-df9c-4542-8788-953a01b63753 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:48.772248 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1215c83a-df9c-4542-8788-953a01b63753 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:48.772674 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1215c83a-df9c-4542-8788-953a01b63753 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:48.773431 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1215c83a-df9c-4542-8788-953a01b63753 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:48.773864 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1215c83a-df9c-4542-8788-953a01b63753 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:48.774285 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1215c83a-df9c-4542-8788-953a01b63753 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:48.923454 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-1215c83a-df9c-4542-8788-953a01b63753 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:48.931817 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-1215c83a-df9c-4542-8788-953a01b63753 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215" status: 200 len: 1318 microversion: 2.1 time: 0.172548 Sep 04 01:06:48.932375 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 136/297] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:48 2022] GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215 => generated 1318 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:49.320885 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-3d0eb801-4b46-4fde-ac6e-c3018ca98a23 tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] Calling method '.version_select at 0x7fc10c764a60>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:49.349534 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.wsgi [None req-3d0eb801-4b46-4fde-ac6e-c3018ca98a23 tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] HTTP exception thrown: Volume 64509ef0-13e6-43d5-a66c-a6cf6bd4d228 could not be found. Sep 04 01:06:49.351472 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-3d0eb801-4b46-4fde-ac6e-c3018ca98a23 tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] Request method failure captured: Sep 04 01:06:49.351472 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: request: GET /compute/v2.1/os-volumes/64509ef0-13e6-43d5-a66c-a6cf6bd4d228 HTTP/1.1 Sep 04 01:06:49.351472 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept: application/json Sep 04 01:06:49.351472 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept-Encoding: identity Sep 04 01:06:49.351472 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Connection: close Sep 04 01:06:49.351472 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Length: 0 Sep 04 01:06:49.351472 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Type: application/json Sep 04 01:06:49.351472 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Host: 10.222.0.42 Sep 04 01:06:49.351472 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Openstack-System-Scope: None Sep 04 01:06:49.351472 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: User-Agent: python-urllib3/1.26.12 Sep 04 01:06:49.351472 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Auth-Token: gAAAAABjE_of3TI7KqM3ooPmOchCj0Ut2Wq36cGVH4FUfdga2xsxLorCG9ZucGeQpVSFwI9tAwUqkuVB1_SMMzJ2QQvCAmxhxK-BuRw4zwxmGU4nd1PeiuFadlv7GNJs6-RsXXfjZUjzvXgI7XhYicagqGeHE-78QAqrfBZWEB2F9CiNIgd5CVM Sep 04 01:06:49.351472 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Id: None Sep 04 01:06:49.351472 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Name: None Sep 04 01:06:49.351472 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Identity-Status: Confirmed Sep 04 01:06:49.351472 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Is-Admin-Project: True Sep 04 01:06:49.351472 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Id: default Sep 04 01:06:49.351472 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Name: Default Sep 04 01:06:49.351472 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Id: 07a7237c4f0c4f3e8e43aa14ec0a9064 Sep 04 01:06:49.351472 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1932271488 Sep 04 01:06:49.351472 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Role: reader,member Sep 04 01:06:49.351472 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Roles: reader,member Sep 04 01:06:49.352904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/07a7237c4f0c4f3e8e43aa14ec0a9064"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_07a7237c4f0c4f3e8e43aa14ec0a9064", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/07a7237c4f0c4f3e8e43aa14ec0a9064"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/07a7237c4f0c4f3e8e43aa14ec0a9064"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:06:49.352904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1932271488 Sep 04 01:06:49.352904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Id: 07a7237c4f0c4f3e8e43aa14ec0a9064 Sep 04 01:06:49.352904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1932271488 Sep 04 01:06:49.352904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User: tempest-VolumesSnapshotsTestJSON-1932271488-project Sep 04 01:06:49.352904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Id: default Sep 04 01:06:49.352904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Name: Default Sep 04 01:06:49.352904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Id: f55f74f18e4c4e77b2d9d8b68ae0ea90 Sep 04 01:06:49.352904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1932271488-project Sep 04 01:06:49.352904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: method: Sep 04 01:06:49.352904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: exception: Volume 64509ef0-13e6-43d5-a66c-a6cf6bd4d228 could not be found. Sep 04 01:06:49.352904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: action_args: {'id': '64509ef0-13e6-43d5-a66c-a6cf6bd4d228'} Sep 04 01:06:49.352904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:06:49.353753 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:06:49.353753 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 04 01:06:49.353753 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 04 01:06:49.353753 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 04 01:06:49.353753 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 04 01:06:49.353753 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 04 01:06:49.353753 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 04 01:06:49.353753 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 04 01:06:49.353753 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 04 01:06:49.353753 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 04 01:06:49.353753 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 04 01:06:49.353753 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 04 01:06:49.353753 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 04 01:06:49.353753 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 04 01:06:49.353753 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 04 01:06:49.353753 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 64509ef0-13e6-43d5-a66c-a6cf6bd4d228 could not be found. (HTTP 404) (Request-ID: req-32be48f8-858f-4eaf-9aec-1793b41791fe) Sep 04 01:06:49.353753 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:06:49.353753 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:06:49.353753 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:06:49.355016 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:06:49.355016 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 04 01:06:49.355016 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 04 01:06:49.355016 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 04 01:06:49.355016 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 04 01:06:49.355016 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 04 01:06:49.355016 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 04 01:06:49.355016 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 04 01:06:49.355016 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 04 01:06:49.355016 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 04 01:06:49.355016 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 04 01:06:49.355016 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 04 01:06:49.355016 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 04 01:06:49.355016 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 04 01:06:49.355016 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 04 01:06:49.355016 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 04 01:06:49.355016 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 04 01:06:49.355016 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 04 01:06:49.355016 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 04 01:06:49.356175 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 04 01:06:49.356175 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 04 01:06:49.356175 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 04 01:06:49.356175 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 04 01:06:49.356175 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 64509ef0-13e6-43d5-a66c-a6cf6bd4d228 could not be found. Sep 04 01:06:49.356175 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:06:49.356175 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:06:49.356175 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:06:49.356175 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:06:49.356175 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:06:49.356175 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:06:49.356175 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:06:49.356175 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:06:49.356175 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 04 01:06:49.356175 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 04 01:06:49.356175 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:06:49.356175 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:06:49.356175 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 04 01:06:49.356175 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:06:49.357302 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 64509ef0-13e6-43d5-a66c-a6cf6bd4d228 could not be found. Sep 04 01:06:49.357302 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:06:49.357302 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:06:49.357302 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:06:49.357302 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:06:49.357302 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:06:49.357302 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:06:49.357302 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:06:49.357302 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:06:49.357302 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 64509ef0-13e6-43d5-a66c-a6cf6bd4d228 could not be found. Sep 04 01:06:49.357302 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:06:49.357302 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-3d0eb801-4b46-4fde-ac6e-c3018ca98a23 tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] Returning 404 to user: Volume 64509ef0-13e6-43d5-a66c-a6cf6bd4d228 could not be found. {{(pid=121036) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:06:49.357302 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-3d0eb801-4b46-4fde-ac6e-c3018ca98a23 tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/64509ef0-13e6-43d5-a66c-a6cf6bd4d228" status: 404 len: 109 microversion: 2.1 time: 0.035840 Sep 04 01:06:49.357302 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 162/298] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Sep 4 01:06:49 2022] GET /compute/v2.1/os-volumes/64509ef0-13e6-43d5-a66c-a6cf6bd4d228 => generated 109 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 04 01:06:49.364318 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-842b59c3-899a-4840-9322-0527b4169fef tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] Calling method '.version_select at 0x7fc10c882790>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:49.394561 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.wsgi [None req-842b59c3-899a-4840-9322-0527b4169fef tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] HTTP exception thrown: Volume 64509ef0-13e6-43d5-a66c-a6cf6bd4d228 could not be found. Sep 04 01:06:49.396540 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-842b59c3-899a-4840-9322-0527b4169fef tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] Request method failure captured: Sep 04 01:06:49.396540 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: request: DELETE /compute/v2.1/os-volumes/64509ef0-13e6-43d5-a66c-a6cf6bd4d228 HTTP/1.1 Sep 04 01:06:49.396540 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept: application/json Sep 04 01:06:49.396540 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept-Encoding: identity Sep 04 01:06:49.396540 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Connection: close Sep 04 01:06:49.396540 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Length: 0 Sep 04 01:06:49.396540 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Type: application/json Sep 04 01:06:49.396540 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Host: 10.222.0.42 Sep 04 01:06:49.396540 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Openstack-System-Scope: None Sep 04 01:06:49.396540 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: User-Agent: python-urllib3/1.26.12 Sep 04 01:06:49.396540 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Auth-Token: gAAAAABjE_of3TI7KqM3ooPmOchCj0Ut2Wq36cGVH4FUfdga2xsxLorCG9ZucGeQpVSFwI9tAwUqkuVB1_SMMzJ2QQvCAmxhxK-BuRw4zwxmGU4nd1PeiuFadlv7GNJs6-RsXXfjZUjzvXgI7XhYicagqGeHE-78QAqrfBZWEB2F9CiNIgd5CVM Sep 04 01:06:49.396540 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Id: None Sep 04 01:06:49.396540 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Name: None Sep 04 01:06:49.396540 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Identity-Status: Confirmed Sep 04 01:06:49.396540 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Is-Admin-Project: True Sep 04 01:06:49.396540 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Id: default Sep 04 01:06:49.396540 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Name: Default Sep 04 01:06:49.396540 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Id: 07a7237c4f0c4f3e8e43aa14ec0a9064 Sep 04 01:06:49.396540 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1932271488 Sep 04 01:06:49.396540 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Role: reader,member Sep 04 01:06:49.396540 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Roles: reader,member Sep 04 01:06:49.397829 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/07a7237c4f0c4f3e8e43aa14ec0a9064"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_07a7237c4f0c4f3e8e43aa14ec0a9064", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/07a7237c4f0c4f3e8e43aa14ec0a9064"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/07a7237c4f0c4f3e8e43aa14ec0a9064"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:06:49.397829 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1932271488 Sep 04 01:06:49.397829 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Id: 07a7237c4f0c4f3e8e43aa14ec0a9064 Sep 04 01:06:49.397829 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1932271488 Sep 04 01:06:49.397829 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User: tempest-VolumesSnapshotsTestJSON-1932271488-project Sep 04 01:06:49.397829 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Id: default Sep 04 01:06:49.397829 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Name: Default Sep 04 01:06:49.397829 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Id: f55f74f18e4c4e77b2d9d8b68ae0ea90 Sep 04 01:06:49.397829 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1932271488-project Sep 04 01:06:49.397829 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: method: Sep 04 01:06:49.397829 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: exception: Volume 64509ef0-13e6-43d5-a66c-a6cf6bd4d228 could not be found. Sep 04 01:06:49.397829 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: action_args: {'id': '64509ef0-13e6-43d5-a66c-a6cf6bd4d228'} Sep 04 01:06:49.397829 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:06:49.399983 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:06:49.399983 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 04 01:06:49.399983 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 04 01:06:49.399983 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 04 01:06:49.399983 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 04 01:06:49.399983 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 04 01:06:49.399983 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 04 01:06:49.399983 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 04 01:06:49.399983 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 04 01:06:49.399983 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 04 01:06:49.399983 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 04 01:06:49.399983 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 04 01:06:49.399983 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 04 01:06:49.399983 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 04 01:06:49.399983 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 04 01:06:49.399983 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 64509ef0-13e6-43d5-a66c-a6cf6bd4d228 could not be found. (HTTP 404) (Request-ID: req-2b7ce1bd-91fe-4e3b-929c-b28ffa71b105) Sep 04 01:06:49.399983 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:06:49.399983 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:06:49.399983 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:06:49.414534 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:06:49.414534 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Sep 04 01:06:49.414534 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Sep 04 01:06:49.414534 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 04 01:06:49.414534 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 04 01:06:49.414534 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 04 01:06:49.414534 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 04 01:06:49.414534 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 04 01:06:49.414534 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 04 01:06:49.414534 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 04 01:06:49.414534 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 04 01:06:49.414534 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 04 01:06:49.414534 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 04 01:06:49.414534 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 04 01:06:49.414534 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 04 01:06:49.414534 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 04 01:06:49.414534 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 04 01:06:49.414534 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 04 01:06:49.414534 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 04 01:06:49.415736 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 04 01:06:49.415736 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 04 01:06:49.415736 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 04 01:06:49.415736 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 04 01:06:49.415736 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 64509ef0-13e6-43d5-a66c-a6cf6bd4d228 could not be found. Sep 04 01:06:49.415736 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:06:49.415736 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:06:49.415736 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:06:49.415736 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:06:49.415736 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:06:49.415736 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:06:49.415736 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:06:49.415736 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:06:49.415736 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 04 01:06:49.415736 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 04 01:06:49.415736 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:06:49.415736 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:06:49.415736 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Sep 04 01:06:49.415736 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:06:49.416941 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 64509ef0-13e6-43d5-a66c-a6cf6bd4d228 could not be found. Sep 04 01:06:49.416941 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:06:49.416941 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:06:49.416941 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:06:49.416941 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:06:49.416941 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:06:49.416941 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:06:49.416941 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:06:49.416941 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:06:49.416941 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 64509ef0-13e6-43d5-a66c-a6cf6bd4d228 could not be found. Sep 04 01:06:49.416941 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:06:49.416941 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-842b59c3-899a-4840-9322-0527b4169fef tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] Returning 404 to user: Volume 64509ef0-13e6-43d5-a66c-a6cf6bd4d228 could not be found. {{(pid=121037) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:06:49.416941 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-842b59c3-899a-4840-9322-0527b4169fef tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] 10.222.0.42 "DELETE /compute/v2.1/os-volumes/64509ef0-13e6-43d5-a66c-a6cf6bd4d228" status: 404 len: 109 microversion: 2.1 time: 0.038511 Sep 04 01:06:49.416941 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 137/299] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Sep 4 01:06:49 2022] DELETE /compute/v2.1/os-volumes/64509ef0-13e6-43d5-a66c-a6cf6bd4d228 => generated 109 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 04 01:06:49.417906 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-c3265301-c5ca-474a-a26d-cc4df68d5dce tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] Calling method '.version_select at 0x7fc10c76d310>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:49.443439 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.wsgi [None req-c3265301-c5ca-474a-a26d-cc4df68d5dce tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] HTTP exception thrown: Volume 64509ef0-13e6-43d5-a66c-a6cf6bd4d228 could not be found. Sep 04 01:06:49.445401 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-c3265301-c5ca-474a-a26d-cc4df68d5dce tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] Request method failure captured: Sep 04 01:06:49.445401 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: request: GET /compute/v2.1/os-volumes/64509ef0-13e6-43d5-a66c-a6cf6bd4d228 HTTP/1.1 Sep 04 01:06:49.445401 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept: application/json Sep 04 01:06:49.445401 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept-Encoding: identity Sep 04 01:06:49.445401 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Connection: close Sep 04 01:06:49.445401 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Length: 0 Sep 04 01:06:49.445401 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Type: application/json Sep 04 01:06:49.445401 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Host: 10.222.0.42 Sep 04 01:06:49.445401 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Openstack-System-Scope: None Sep 04 01:06:49.445401 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: User-Agent: python-urllib3/1.26.12 Sep 04 01:06:49.445401 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Auth-Token: gAAAAABjE_of3TI7KqM3ooPmOchCj0Ut2Wq36cGVH4FUfdga2xsxLorCG9ZucGeQpVSFwI9tAwUqkuVB1_SMMzJ2QQvCAmxhxK-BuRw4zwxmGU4nd1PeiuFadlv7GNJs6-RsXXfjZUjzvXgI7XhYicagqGeHE-78QAqrfBZWEB2F9CiNIgd5CVM Sep 04 01:06:49.445401 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Id: None Sep 04 01:06:49.445401 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Name: None Sep 04 01:06:49.445401 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Identity-Status: Confirmed Sep 04 01:06:49.445401 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Is-Admin-Project: True Sep 04 01:06:49.445401 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Id: default Sep 04 01:06:49.445401 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Name: Default Sep 04 01:06:49.445401 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Id: 07a7237c4f0c4f3e8e43aa14ec0a9064 Sep 04 01:06:49.445401 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1932271488 Sep 04 01:06:49.445401 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Role: reader,member Sep 04 01:06:49.445401 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Roles: reader,member Sep 04 01:06:49.446749 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/07a7237c4f0c4f3e8e43aa14ec0a9064"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_07a7237c4f0c4f3e8e43aa14ec0a9064", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/07a7237c4f0c4f3e8e43aa14ec0a9064"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/07a7237c4f0c4f3e8e43aa14ec0a9064"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:06:49.446749 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1932271488 Sep 04 01:06:49.446749 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Id: 07a7237c4f0c4f3e8e43aa14ec0a9064 Sep 04 01:06:49.446749 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1932271488 Sep 04 01:06:49.446749 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User: tempest-VolumesSnapshotsTestJSON-1932271488-project Sep 04 01:06:49.446749 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Id: default Sep 04 01:06:49.446749 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Name: Default Sep 04 01:06:49.446749 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Id: f55f74f18e4c4e77b2d9d8b68ae0ea90 Sep 04 01:06:49.446749 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1932271488-project Sep 04 01:06:49.446749 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: method: Sep 04 01:06:49.446749 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: exception: Volume 64509ef0-13e6-43d5-a66c-a6cf6bd4d228 could not be found. Sep 04 01:06:49.446749 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: action_args: {'id': '64509ef0-13e6-43d5-a66c-a6cf6bd4d228'} Sep 04 01:06:49.446749 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:06:49.447545 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:06:49.447545 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 04 01:06:49.447545 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 04 01:06:49.447545 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 04 01:06:49.447545 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 04 01:06:49.447545 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 04 01:06:49.447545 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 04 01:06:49.447545 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 04 01:06:49.447545 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 04 01:06:49.447545 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 04 01:06:49.447545 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 04 01:06:49.447545 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 04 01:06:49.447545 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 04 01:06:49.447545 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 04 01:06:49.447545 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 04 01:06:49.447545 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 64509ef0-13e6-43d5-a66c-a6cf6bd4d228 could not be found. (HTTP 404) (Request-ID: req-967efafc-8bc2-4f4d-91f8-c4ad3b18cfd5) Sep 04 01:06:49.447545 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:06:49.447545 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:06:49.447545 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:06:49.448856 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:06:49.448856 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 04 01:06:49.448856 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 04 01:06:49.448856 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 04 01:06:49.448856 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 04 01:06:49.448856 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 04 01:06:49.448856 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 04 01:06:49.448856 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 04 01:06:49.448856 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 04 01:06:49.448856 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 04 01:06:49.448856 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 04 01:06:49.448856 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 04 01:06:49.448856 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 04 01:06:49.448856 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 04 01:06:49.448856 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 04 01:06:49.448856 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 04 01:06:49.448856 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 04 01:06:49.448856 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 04 01:06:49.448856 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 04 01:06:49.450145 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 04 01:06:49.450145 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 04 01:06:49.450145 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 04 01:06:49.450145 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 04 01:06:49.450145 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 64509ef0-13e6-43d5-a66c-a6cf6bd4d228 could not be found. Sep 04 01:06:49.450145 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:06:49.450145 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:06:49.450145 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:06:49.450145 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:06:49.450145 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:06:49.450145 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:06:49.450145 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:06:49.450145 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:06:49.450145 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 04 01:06:49.450145 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 04 01:06:49.450145 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:06:49.450145 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:06:49.450145 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 04 01:06:49.450145 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:06:49.451276 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 64509ef0-13e6-43d5-a66c-a6cf6bd4d228 could not be found. Sep 04 01:06:49.451276 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:06:49.451276 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:06:49.451276 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:06:49.451276 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:06:49.451276 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:06:49.451276 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:06:49.451276 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:06:49.451276 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:06:49.451276 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 64509ef0-13e6-43d5-a66c-a6cf6bd4d228 could not be found. Sep 04 01:06:49.451276 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:06:49.451276 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-c3265301-c5ca-474a-a26d-cc4df68d5dce tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] Returning 404 to user: Volume 64509ef0-13e6-43d5-a66c-a6cf6bd4d228 could not be found. {{(pid=121036) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:06:49.451276 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-c3265301-c5ca-474a-a26d-cc4df68d5dce tempest-VolumesSnapshotsTestJSON-1932271488 tempest-VolumesSnapshotsTestJSON-1932271488-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/64509ef0-13e6-43d5-a66c-a6cf6bd4d228" status: 404 len: 109 microversion: 2.1 time: 0.043949 Sep 04 01:06:49.451276 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 163/300] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Sep 4 01:06:49 2022] GET /compute/v2.1/os-volumes/64509ef0-13e6-43d5-a66c-a6cf6bd4d228 => generated 109 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 04 01:06:49.603928 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-545b4dd0-9892-4769-ac64-a0e2f0f0eae2 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:49.604530 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-545b4dd0-9892-4769-ac64-a0e2f0f0eae2 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:49.613709 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-545b4dd0-9892-4769-ac64-a0e2f0f0eae2 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:49.614244 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-545b4dd0-9892-4769-ac64-a0e2f0f0eae2 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:49.614699 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-545b4dd0-9892-4769-ac64-a0e2f0f0eae2 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:49.615391 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-545b4dd0-9892-4769-ac64-a0e2f0f0eae2 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:49.615836 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-545b4dd0-9892-4769-ac64-a0e2f0f0eae2 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:49.616255 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-545b4dd0-9892-4769-ac64-a0e2f0f0eae2 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:49.761902 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-545b4dd0-9892-4769-ac64-a0e2f0f0eae2 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:49.770466 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-545b4dd0-9892-4769-ac64-a0e2f0f0eae2 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1499 microversion: 2.1 time: 0.170454 Sep 04 01:06:49.771249 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 138/301] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:49 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1499 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:49.794632 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-6abc6f4b-58ab-44a9-bdc2-2b896c89856a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:49.795277 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-6abc6f4b-58ab-44a9-bdc2-2b896c89856a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:49.797235 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell0:SELECT=8 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:49.802030 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6abc6f4b-58ab-44a9-bdc2-2b896c89856a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:49.802514 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6abc6f4b-58ab-44a9-bdc2-2b896c89856a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:49.802958 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6abc6f4b-58ab-44a9-bdc2-2b896c89856a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:49.803646 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6abc6f4b-58ab-44a9-bdc2-2b896c89856a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:49.804099 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6abc6f4b-58ab-44a9-bdc2-2b896c89856a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:49.804578 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6abc6f4b-58ab-44a9-bdc2-2b896c89856a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:49.942650 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-6abc6f4b-58ab-44a9-bdc2-2b896c89856a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:49.952140 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-6abc6f4b-58ab-44a9-bdc2-2b896c89856a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1516 microversion: 2.1 time: 0.161277 Sep 04 01:06:49.953108 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 164/302] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:49 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1516 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:49.955077 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-3cffffa2-0ff6-45d5-b460-56f42355620e tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:49.955769 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-3cffffa2-0ff6-45d5-b460-56f42355620e tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: 2808d334-c39c-4a09-88c7-c1711cce4215] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:49.961857 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3cffffa2-0ff6-45d5-b460-56f42355620e tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:49.962335 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3cffffa2-0ff6-45d5-b460-56f42355620e tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:49.962959 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3cffffa2-0ff6-45d5-b460-56f42355620e tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:49.963597 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3cffffa2-0ff6-45d5-b460-56f42355620e tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:49.964267 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3cffffa2-0ff6-45d5-b460-56f42355620e tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:49.964489 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3cffffa2-0ff6-45d5-b460-56f42355620e tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:50.126128 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-3cffffa2-0ff6-45d5-b460-56f42355620e tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:50.135562 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-3cffffa2-0ff6-45d5-b460-56f42355620e tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215" status: 200 len: 1318 microversion: 2.1 time: 0.184473 Sep 04 01:06:50.136127 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 139/303] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:49 2022] GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215 => generated 1318 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:06:51.158245 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-7691e59c-7f63-41b6-97f6-370be8f3f7ce tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:51.159190 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-7691e59c-7f63-41b6-97f6-370be8f3f7ce tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: 2808d334-c39c-4a09-88c7-c1711cce4215] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:51.165466 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7691e59c-7f63-41b6-97f6-370be8f3f7ce tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:51.165971 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7691e59c-7f63-41b6-97f6-370be8f3f7ce tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:51.169584 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7691e59c-7f63-41b6-97f6-370be8f3f7ce tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:51.169584 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7691e59c-7f63-41b6-97f6-370be8f3f7ce tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:51.169584 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7691e59c-7f63-41b6-97f6-370be8f3f7ce tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:51.169584 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7691e59c-7f63-41b6-97f6-370be8f3f7ce tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:51.301200 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-7691e59c-7f63-41b6-97f6-370be8f3f7ce tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:51.308777 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-7691e59c-7f63-41b6-97f6-370be8f3f7ce tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215" status: 200 len: 1318 microversion: 2.1 time: 0.154480 Sep 04 01:06:51.309357 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 165/304] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:51 2022] GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215 => generated 1318 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:51.805420 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-d0e7a0c5-8a41-4c85-b4fa-8ad52ae4588e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:06:51.807829 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-d0e7a0c5-8a41-4c85-b4fa-8ad52ae4588e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:51.813924 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d0e7a0c5-8a41-4c85-b4fa-8ad52ae4588e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:51.814477 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d0e7a0c5-8a41-4c85-b4fa-8ad52ae4588e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:51.815156 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d0e7a0c5-8a41-4c85-b4fa-8ad52ae4588e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:51.816137 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d0e7a0c5-8a41-4c85-b4fa-8ad52ae4588e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:51.816660 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d0e7a0c5-8a41-4c85-b4fa-8ad52ae4588e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:51.817210 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d0e7a0c5-8a41-4c85-b4fa-8ad52ae4588e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:51.841894 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.objects.instance [None req-d0e7a0c5-8a41-4c85-b4fa-8ad52ae4588e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lazy-loading 'flavor' on Instance uuid 2f7a7e72-1f16-4ab4-b826-d376d39be4c2 {{(pid=121037) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:06:52.006733 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=2 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:52.182017 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-d0e7a0c5-8a41-4c85-b4fa-8ad52ae4588e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "POST /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2/action" status: 200 len: 32 microversion: 2.1 time: 0.380858 Sep 04 01:06:52.182626 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 140/305] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Sep 4 01:06:51 2022] POST /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2/action => generated 32 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 04 01:06:52.192471 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-6b0209d8-da73-4e33-84ba-d0a1e5401708 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:52.193089 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-6b0209d8-da73-4e33-84ba-d0a1e5401708 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:52.199495 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6b0209d8-da73-4e33-84ba-d0a1e5401708 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:52.199986 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6b0209d8-da73-4e33-84ba-d0a1e5401708 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:52.200419 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6b0209d8-da73-4e33-84ba-d0a1e5401708 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:52.204700 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6b0209d8-da73-4e33-84ba-d0a1e5401708 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:52.204700 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6b0209d8-da73-4e33-84ba-d0a1e5401708 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:52.204700 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6b0209d8-da73-4e33-84ba-d0a1e5401708 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:52.333463 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-3209a7b7-078d-4e4b-84a3-2b64d7d7a0c7 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:52.334169 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-3209a7b7-078d-4e4b-84a3-2b64d7d7a0c7 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: 2808d334-c39c-4a09-88c7-c1711cce4215] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:52.342300 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3209a7b7-078d-4e4b-84a3-2b64d7d7a0c7 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:52.342779 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3209a7b7-078d-4e4b-84a3-2b64d7d7a0c7 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:52.343381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3209a7b7-078d-4e4b-84a3-2b64d7d7a0c7 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:52.344222 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3209a7b7-078d-4e4b-84a3-2b64d7d7a0c7 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:52.344675 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3209a7b7-078d-4e4b-84a3-2b64d7d7a0c7 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:52.345127 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3209a7b7-078d-4e4b-84a3-2b64d7d7a0c7 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:52.354525 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-6b0209d8-da73-4e33-84ba-d0a1e5401708 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:52.366168 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-6b0209d8-da73-4e33-84ba-d0a1e5401708 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1522 microversion: 2.1 time: 0.177874 Sep 04 01:06:52.367062 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 166/306] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:52 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1522 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:52.589887 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-3209a7b7-078d-4e4b-84a3-2b64d7d7a0c7 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:52.597667 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-3209a7b7-078d-4e4b-84a3-2b64d7d7a0c7 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215" status: 200 len: 1318 microversion: 2.1 time: 0.269813 Sep 04 01:06:52.598228 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 141/307] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:52 2022] GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215 => generated 1318 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:06:53.393310 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-fa302c56-a425-4577-8baa-ff4dbbef27ce tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:53.393820 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-fa302c56-a425-4577-8baa-ff4dbbef27ce tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:53.399636 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fa302c56-a425-4577-8baa-ff4dbbef27ce tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:53.400136 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fa302c56-a425-4577-8baa-ff4dbbef27ce tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:53.400566 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fa302c56-a425-4577-8baa-ff4dbbef27ce tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:53.401274 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fa302c56-a425-4577-8baa-ff4dbbef27ce tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:53.401720 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fa302c56-a425-4577-8baa-ff4dbbef27ce tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:53.402136 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fa302c56-a425-4577-8baa-ff4dbbef27ce tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:53.405945 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=100 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:53.525769 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-82352eab-4a9e-4ed7-90b5-d62da10059f8 tempest-VolumesTestJSON-1283084924 tempest-VolumesTestJSON-1283084924-project] Calling method '.version_select at 0x7fc10c9b1ee0>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:53.530246 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-fa302c56-a425-4577-8baa-ff4dbbef27ce tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:53.539008 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-fa302c56-a425-4577-8baa-ff4dbbef27ce tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1522 microversion: 2.1 time: 0.149903 Sep 04 01:06:53.539567 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 167/308] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:53 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1522 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:53.572592 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-82352eab-4a9e-4ed7-90b5-d62da10059f8 tempest-VolumesTestJSON-1283084924 tempest-VolumesTestJSON-1283084924-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/83a4278a-041d-4054-8417-20c6138f4c63" status: 200 len: 351 microversion: 2.1 time: 0.050504 Sep 04 01:06:53.573268 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 142/309] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Sep 4 01:06:53 2022] GET /compute/v2.1/os-volumes/83a4278a-041d-4054-8417-20c6138f4c63 => generated 351 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:06:53.621432 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-50d4db18-992a-4697-9cea-2cf59d10258e tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:53.622025 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-50d4db18-992a-4697-9cea-2cf59d10258e tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: 2808d334-c39c-4a09-88c7-c1711cce4215] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:53.628469 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-50d4db18-992a-4697-9cea-2cf59d10258e tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:53.628950 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-50d4db18-992a-4697-9cea-2cf59d10258e tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:53.629484 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-50d4db18-992a-4697-9cea-2cf59d10258e tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:53.630335 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-50d4db18-992a-4697-9cea-2cf59d10258e tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:53.630612 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-50d4db18-992a-4697-9cea-2cf59d10258e tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:53.631040 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-50d4db18-992a-4697-9cea-2cf59d10258e tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:53.769370 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-50d4db18-992a-4697-9cea-2cf59d10258e tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:53.777036 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-50d4db18-992a-4697-9cea-2cf59d10258e tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215" status: 200 len: 1318 microversion: 2.1 time: 0.159332 Sep 04 01:06:53.777624 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 168/310] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:53 2022] GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215 => generated 1318 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:54.563778 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-28cf8e8d-6f7c-4e7f-b66e-4ef5bf40ce4c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:54.565138 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-28cf8e8d-6f7c-4e7f-b66e-4ef5bf40ce4c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:54.571806 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-28cf8e8d-6f7c-4e7f-b66e-4ef5bf40ce4c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:54.572277 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-28cf8e8d-6f7c-4e7f-b66e-4ef5bf40ce4c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:54.572702 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-28cf8e8d-6f7c-4e7f-b66e-4ef5bf40ce4c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:54.573404 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-28cf8e8d-6f7c-4e7f-b66e-4ef5bf40ce4c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:54.573849 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-28cf8e8d-6f7c-4e7f-b66e-4ef5bf40ce4c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:54.574267 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-28cf8e8d-6f7c-4e7f-b66e-4ef5bf40ce4c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:54.707424 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-28cf8e8d-6f7c-4e7f-b66e-4ef5bf40ce4c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:54.715529 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-28cf8e8d-6f7c-4e7f-b66e-4ef5bf40ce4c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1522 microversion: 2.1 time: 0.156230 Sep 04 01:06:54.716235 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 143/311] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:54 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1522 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:54.801927 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-eefa8170-0ab8-4994-8260-157da14224c9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:54.802527 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-eefa8170-0ab8-4994-8260-157da14224c9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: 2808d334-c39c-4a09-88c7-c1711cce4215] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:54.808559 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-eefa8170-0ab8-4994-8260-157da14224c9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:54.808991 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-eefa8170-0ab8-4994-8260-157da14224c9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:54.809483 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-eefa8170-0ab8-4994-8260-157da14224c9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:54.813813 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-eefa8170-0ab8-4994-8260-157da14224c9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:54.813813 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-eefa8170-0ab8-4994-8260-157da14224c9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:54.813813 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-eefa8170-0ab8-4994-8260-157da14224c9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:54.941375 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-eefa8170-0ab8-4994-8260-157da14224c9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:54.949069 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-eefa8170-0ab8-4994-8260-157da14224c9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215" status: 200 len: 1318 microversion: 2.1 time: 0.151784 Sep 04 01:06:54.949676 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 169/312] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:54 2022] GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215 => generated 1318 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:55.202871 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-3f45a1fd-2cc5-40dd-a93e-8c83bf6453cc tempest-VolumesTestJSON-1283084924 tempest-VolumesTestJSON-1283084924-project] Calling method '.version_select at 0x7fc10c9b1ee0>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:55.251108 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-3f45a1fd-2cc5-40dd-a93e-8c83bf6453cc tempest-VolumesTestJSON-1283084924 tempest-VolumesTestJSON-1283084924-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/e8e8677e-b3ba-437f-bf15-89c630fea140" status: 200 len: 350 microversion: 2.1 time: 0.051892 Sep 04 01:06:55.251671 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 144/313] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Sep 4 01:06:55 2022] GET /compute/v2.1/os-volumes/e8e8677e-b3ba-437f-bf15-89c630fea140 => generated 350 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:06:55.739334 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-1a8d90ff-7fc1-4940-912a-2bd95b8546a9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:55.739943 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-1a8d90ff-7fc1-4940-912a-2bd95b8546a9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:55.746337 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1a8d90ff-7fc1-4940-912a-2bd95b8546a9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:55.746824 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1a8d90ff-7fc1-4940-912a-2bd95b8546a9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:55.747387 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1a8d90ff-7fc1-4940-912a-2bd95b8546a9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:55.748328 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1a8d90ff-7fc1-4940-912a-2bd95b8546a9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:55.748782 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1a8d90ff-7fc1-4940-912a-2bd95b8546a9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:55.749245 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1a8d90ff-7fc1-4940-912a-2bd95b8546a9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:55.885476 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-1a8d90ff-7fc1-4940-912a-2bd95b8546a9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:55.896627 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-1a8d90ff-7fc1-4940-912a-2bd95b8546a9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1522 microversion: 2.1 time: 0.159047 Sep 04 01:06:55.896627 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 170/314] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:55 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1522 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:55.976654 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-3487b0ec-e31b-459e-b9d0-3c68e107cb96 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:55.977294 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-3487b0ec-e31b-459e-b9d0-3c68e107cb96 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: 2808d334-c39c-4a09-88c7-c1711cce4215] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:55.984626 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3487b0ec-e31b-459e-b9d0-3c68e107cb96 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:55.985469 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3487b0ec-e31b-459e-b9d0-3c68e107cb96 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:55.985969 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3487b0ec-e31b-459e-b9d0-3c68e107cb96 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:55.987490 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3487b0ec-e31b-459e-b9d0-3c68e107cb96 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:55.988209 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3487b0ec-e31b-459e-b9d0-3c68e107cb96 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:55.988585 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3487b0ec-e31b-459e-b9d0-3c68e107cb96 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:56.124383 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-3487b0ec-e31b-459e-b9d0-3c68e107cb96 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:56.132187 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-3487b0ec-e31b-459e-b9d0-3c68e107cb96 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215" status: 200 len: 1318 microversion: 2.1 time: 0.160107 Sep 04 01:06:56.132762 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 145/315] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:55 2022] GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215 => generated 1318 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:56.167373 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-40c50cc1-b0fc-42a7-ab44-7b46197ba9c7 req-f2b60c61-ef6b-453a-b62a-083a020af4f4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2808d334-c39c-4a09-88c7-c1711cce4215", "name": "network-vif-plugged", "status": "completed", "tag": "449add0b-213e-49af-b0f2-5656f19d796f"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:06:56.176681 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-40c50cc1-b0fc-42a7-ab44-7b46197ba9c7 req-f2b60c61-ef6b-453a-b62a-083a020af4f4 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:56.177207 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-40c50cc1-b0fc-42a7-ab44-7b46197ba9c7 req-f2b60c61-ef6b-453a-b62a-083a020af4f4 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:56.177635 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-40c50cc1-b0fc-42a7-ab44-7b46197ba9c7 req-f2b60c61-ef6b-453a-b62a-083a020af4f4 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:56.188393 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-40c50cc1-b0fc-42a7-ab44-7b46197ba9c7 req-f2b60c61-ef6b-453a-b62a-083a020af4f4 service nova] Creating event network-vif-plugged:449add0b-213e-49af-b0f2-5656f19d796f for instance 2808d334-c39c-4a09-88c7-c1711cce4215 on hv-cinder-maste Sep 04 01:06:56.193498 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-40c50cc1-b0fc-42a7-ab44-7b46197ba9c7 req-f2b60c61-ef6b-453a-b62a-083a020af4f4 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.029806 Sep 04 01:06:56.194063 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 171/316] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:06:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:06:56.868978 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-45063f25-4692-4765-a4fe-e7e12a8868bf tempest-VolumesTestJSON-1283084924 tempest-VolumesTestJSON-1283084924-project] Calling method '.version_select at 0x7fc10c9884c0>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:56.916294 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-45063f25-4692-4765-a4fe-e7e12a8868bf tempest-VolumesTestJSON-1283084924 tempest-VolumesTestJSON-1283084924-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/74c1d936-ee32-46cf-afcf-6e884b20c9ad" status: 200 len: 351 microversion: 2.1 time: 0.051222 Sep 04 01:06:56.917042 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 146/317] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Sep 4 01:06:56 2022] GET /compute/v2.1/os-volumes/74c1d936-ee32-46cf-afcf-6e884b20c9ad => generated 351 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:06:56.917483 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-458c0bd9-0ff3-4bfa-991c-5df42ab29e2a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:56.920255 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-458c0bd9-0ff3-4bfa-991c-5df42ab29e2a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:56.924771 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-458c0bd9-0ff3-4bfa-991c-5df42ab29e2a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:56.925354 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-458c0bd9-0ff3-4bfa-991c-5df42ab29e2a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:56.925852 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-458c0bd9-0ff3-4bfa-991c-5df42ab29e2a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:56.926540 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-458c0bd9-0ff3-4bfa-991c-5df42ab29e2a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:56.927142 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-458c0bd9-0ff3-4bfa-991c-5df42ab29e2a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:56.927429 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-458c0bd9-0ff3-4bfa-991c-5df42ab29e2a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:56.930553 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-f326cc1f-fb29-437e-8faa-161a2bfee8f2 tempest-VolumesTestJSON-1283084924 tempest-VolumesTestJSON-1283084924-project] Calling method '.version_select at 0x7fc10c84cca0>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:57.016667 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-f326cc1f-fb29-437e-8faa-161a2bfee8f2 tempest-VolumesTestJSON-1283084924 tempest-VolumesTestJSON-1283084924-project] 10.222.0.42 "GET /compute/v2.1/os-volumes" status: 200 len: 1035 microversion: 2.1 time: 0.089810 Sep 04 01:06:57.017442 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 147/318] 10.222.0.42 () {58 vars in 1134 bytes} [Sun Sep 4 01:06:56 2022] GET /compute/v2.1/os-volumes => generated 1035 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:06:57.031150 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-49297433-ae7e-46a1-8353-5ebd3b86ab82 tempest-VolumesTestJSON-1283084924 tempest-VolumesTestJSON-1283084924-project] Calling method '.version_select at 0x7fc10c7bd1f0>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:57.066971 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-458c0bd9-0ff3-4bfa-991c-5df42ab29e2a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:57.074660 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-458c0bd9-0ff3-4bfa-991c-5df42ab29e2a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1522 microversion: 2.1 time: 0.161504 Sep 04 01:06:57.075245 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 172/319] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:56 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1522 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:06:57.119547 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-49297433-ae7e-46a1-8353-5ebd3b86ab82 tempest-VolumesTestJSON-1283084924 tempest-VolumesTestJSON-1283084924-project] 10.222.0.42 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.092120 Sep 04 01:06:57.120120 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 148/320] 10.222.0.42 () {58 vars in 1149 bytes} [Sun Sep 4 01:06:57 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 694 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:06:57.133751 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-170dc696-096b-49a7-b91e-7f8a12ec957c tempest-VolumesTestJSON-1283084924 tempest-VolumesTestJSON-1283084924-project] Calling method '.version_select at 0x7fc10c7644c0>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:57.155525 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-c23d45b0-19b5-4072-b51f-85fb71f6e2f2 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:57.156134 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-c23d45b0-19b5-4072-b51f-85fb71f6e2f2 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: 2808d334-c39c-4a09-88c7-c1711cce4215] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:57.163276 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c23d45b0-19b5-4072-b51f-85fb71f6e2f2 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:57.163762 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c23d45b0-19b5-4072-b51f-85fb71f6e2f2 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:57.164199 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c23d45b0-19b5-4072-b51f-85fb71f6e2f2 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:57.164887 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c23d45b0-19b5-4072-b51f-85fb71f6e2f2 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:57.165439 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c23d45b0-19b5-4072-b51f-85fb71f6e2f2 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:57.165883 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c23d45b0-19b5-4072-b51f-85fb71f6e2f2 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:57.216966 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-170dc696-096b-49a7-b91e-7f8a12ec957c tempest-VolumesTestJSON-1283084924 tempest-VolumesTestJSON-1283084924-project] 10.222.0.42 "GET /compute/v2.1/os-volumes" status: 200 len: 1035 microversion: 2.1 time: 0.086891 Sep 04 01:06:57.217783 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 173/321] 10.222.0.42 () {58 vars in 1134 bytes} [Sun Sep 4 01:06:57 2022] GET /compute/v2.1/os-volumes => generated 1035 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:57.230937 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-4c7db7f2-a75e-483e-8ef4-7484f4b57af0 tempest-VolumesTestJSON-1283084924 tempest-VolumesTestJSON-1283084924-project] Calling method '.version_select at 0x7fc10d41ef70>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:57.301786 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-c23d45b0-19b5-4072-b51f-85fb71f6e2f2 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:57.309481 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-c23d45b0-19b5-4072-b51f-85fb71f6e2f2 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215" status: 200 len: 1318 microversion: 2.1 time: 0.157799 Sep 04 01:06:57.310047 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 149/322] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:57 2022] GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215 => generated 1318 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:06:57.313420 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-4c7db7f2-a75e-483e-8ef4-7484f4b57af0 tempest-VolumesTestJSON-1283084924 tempest-VolumesTestJSON-1283084924-project] 10.222.0.42 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 353 microversion: 2.1 time: 0.086072 Sep 04 01:06:57.313988 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 174/323] 10.222.0.42 () {58 vars in 1167 bytes} [Sun Sep 4 01:06:57 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 353 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:06:57.327219 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-e74b3c18-e0c6-4ea8-af2f-6a5fbcda0d73 tempest-VolumesTestJSON-1283084924 tempest-VolumesTestJSON-1283084924-project] Calling method '.version_select at 0x7fc10c846820>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:57.409042 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-e74b3c18-e0c6-4ea8-af2f-6a5fbcda0d73 tempest-VolumesTestJSON-1283084924 tempest-VolumesTestJSON-1283084924-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.084793 Sep 04 01:06:57.409640 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 150/324] 10.222.0.42 () {58 vars in 1170 bytes} [Sun Sep 4 01:06:57 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 694 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:06:57.423108 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-12320985-cd4c-42a1-a298-1e465bed9aab tempest-VolumesTestJSON-1283084924 tempest-VolumesTestJSON-1283084924-project] Calling method '.version_select at 0x7fc10c6fcca0>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:57.505433 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-12320985-cd4c-42a1-a298-1e465bed9aab tempest-VolumesTestJSON-1283084924 tempest-VolumesTestJSON-1283084924-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1035 microversion: 2.1 time: 0.085953 Sep 04 01:06:57.506701 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 175/325] 10.222.0.42 () {58 vars in 1155 bytes} [Sun Sep 4 01:06:57 2022] GET /compute/v2.1/os-volumes/detail => generated 1035 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:57.519542 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-971782e5-2874-4933-85a3-934eac1539c4 tempest-VolumesTestJSON-1283084924 tempest-VolumesTestJSON-1283084924-project] Calling method '.version_select at 0x7fc10c84c8b0>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:57.604299 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-971782e5-2874-4933-85a3-934eac1539c4 tempest-VolumesTestJSON-1283084924 tempest-VolumesTestJSON-1283084924-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 353 microversion: 2.1 time: 0.088744 Sep 04 01:06:57.604966 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 151/326] 10.222.0.42 () {58 vars in 1188 bytes} [Sun Sep 4 01:06:57 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 353 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:06:57.619092 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-5d56f7dd-7379-459b-a330-a3a9a35ecad6 tempest-VolumesTestJSON-1283084924 tempest-VolumesTestJSON-1283084924-project] Calling method '.version_select at 0x7fc10c741f70>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:57.830612 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-5d56f7dd-7379-459b-a330-a3a9a35ecad6 tempest-VolumesTestJSON-1283084924 tempest-VolumesTestJSON-1283084924-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1035 microversion: 2.1 time: 0.215176 Sep 04 01:06:57.831326 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 176/327] 10.222.0.42 () {58 vars in 1155 bytes} [Sun Sep 4 01:06:57 2022] GET /compute/v2.1/os-volumes/detail => generated 1035 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:58.107376 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-06063e1a-1cfb-47de-b630-c2933b4ca598 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:58.107889 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-06063e1a-1cfb-47de-b630-c2933b4ca598 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:58.114797 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-06063e1a-1cfb-47de-b630-c2933b4ca598 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:58.115274 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-06063e1a-1cfb-47de-b630-c2933b4ca598 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:58.115704 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-06063e1a-1cfb-47de-b630-c2933b4ca598 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:58.116391 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-06063e1a-1cfb-47de-b630-c2933b4ca598 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:58.116832 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-06063e1a-1cfb-47de-b630-c2933b4ca598 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:58.117273 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-06063e1a-1cfb-47de-b630-c2933b4ca598 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:58.246294 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-06063e1a-1cfb-47de-b630-c2933b4ca598 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:58.255019 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-06063e1a-1cfb-47de-b630-c2933b4ca598 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1522 microversion: 2.1 time: 0.153149 Sep 04 01:06:58.255581 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 152/328] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:58 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1522 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:58.332698 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-98627899-db68-4473-a791-19210ccfb9d9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:58.333344 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-98627899-db68-4473-a791-19210ccfb9d9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: 2808d334-c39c-4a09-88c7-c1711cce4215] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:58.339444 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-98627899-db68-4473-a791-19210ccfb9d9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:58.339919 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-98627899-db68-4473-a791-19210ccfb9d9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:58.340350 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-98627899-db68-4473-a791-19210ccfb9d9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:58.341034 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-98627899-db68-4473-a791-19210ccfb9d9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:58.341505 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-98627899-db68-4473-a791-19210ccfb9d9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:58.341927 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-98627899-db68-4473-a791-19210ccfb9d9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:58.469162 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-98627899-db68-4473-a791-19210ccfb9d9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:58.477132 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-98627899-db68-4473-a791-19210ccfb9d9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215" status: 200 len: 1318 microversion: 2.1 time: 0.148551 Sep 04 01:06:58.477698 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 177/329] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:58 2022] GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215 => generated 1318 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:59.278681 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-d014efd6-2eb8-4552-88e6-5ce496cde5fb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:59.279287 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-d014efd6-2eb8-4552-88e6-5ce496cde5fb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:59.282081 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_api:SELECT=85,nova_api:UPDATE=1 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:06:59.286954 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d014efd6-2eb8-4552-88e6-5ce496cde5fb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:59.287428 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d014efd6-2eb8-4552-88e6-5ce496cde5fb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:59.287856 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d014efd6-2eb8-4552-88e6-5ce496cde5fb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:59.288531 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d014efd6-2eb8-4552-88e6-5ce496cde5fb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:59.289023 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d014efd6-2eb8-4552-88e6-5ce496cde5fb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:59.289469 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d014efd6-2eb8-4552-88e6-5ce496cde5fb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:59.419281 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-d014efd6-2eb8-4552-88e6-5ce496cde5fb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:59.427208 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-d014efd6-2eb8-4552-88e6-5ce496cde5fb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1522 microversion: 2.1 time: 0.152343 Sep 04 01:06:59.427779 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 153/330] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:59 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1522 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:06:59.500933 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-d85c69eb-4e65-41be-afa0-859548b220f8 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:06:59.501571 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-d85c69eb-4e65-41be-afa0-859548b220f8 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: 2808d334-c39c-4a09-88c7-c1711cce4215] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:06:59.507984 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d85c69eb-4e65-41be-afa0-859548b220f8 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:59.508458 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d85c69eb-4e65-41be-afa0-859548b220f8 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:59.508884 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d85c69eb-4e65-41be-afa0-859548b220f8 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:59.509628 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d85c69eb-4e65-41be-afa0-859548b220f8 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:06:59.510077 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d85c69eb-4e65-41be-afa0-859548b220f8 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:06:59.510498 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d85c69eb-4e65-41be-afa0-859548b220f8 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:06:59.637359 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-d85c69eb-4e65-41be-afa0-859548b220f8 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:06:59.645160 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-d85c69eb-4e65-41be-afa0-859548b220f8 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215" status: 200 len: 1318 microversion: 2.1 time: 0.148125 Sep 04 01:06:59.645720 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 178/331] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:06:59 2022] GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215 => generated 1318 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:00.451060 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-86eb8840-dc73-4a58-8ac8-696848258a40 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:00.451586 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-86eb8840-dc73-4a58-8ac8-696848258a40 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:00.458183 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-86eb8840-dc73-4a58-8ac8-696848258a40 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:00.458664 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-86eb8840-dc73-4a58-8ac8-696848258a40 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:00.459094 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-86eb8840-dc73-4a58-8ac8-696848258a40 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:00.459780 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-86eb8840-dc73-4a58-8ac8-696848258a40 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:00.460236 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-86eb8840-dc73-4a58-8ac8-696848258a40 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:00.460659 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-86eb8840-dc73-4a58-8ac8-696848258a40 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:00.584860 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-86eb8840-dc73-4a58-8ac8-696848258a40 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:00.592499 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-86eb8840-dc73-4a58-8ac8-696848258a40 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1522 microversion: 2.1 time: 0.145477 Sep 04 01:07:00.593292 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 154/332] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:00 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1522 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:00.668245 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-4e1c388c-e184-4242-a0f7-6380307d5f4a tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:00.668856 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-4e1c388c-e184-4242-a0f7-6380307d5f4a tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: 2808d334-c39c-4a09-88c7-c1711cce4215] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:00.676684 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4e1c388c-e184-4242-a0f7-6380307d5f4a tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:00.677190 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4e1c388c-e184-4242-a0f7-6380307d5f4a tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:00.677626 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4e1c388c-e184-4242-a0f7-6380307d5f4a tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:00.678317 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4e1c388c-e184-4242-a0f7-6380307d5f4a tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:00.678766 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4e1c388c-e184-4242-a0f7-6380307d5f4a tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:00.679189 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4e1c388c-e184-4242-a0f7-6380307d5f4a tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:00.810976 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-4e1c388c-e184-4242-a0f7-6380307d5f4a tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:00.818833 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-4e1c388c-e184-4242-a0f7-6380307d5f4a tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215" status: 200 len: 1318 microversion: 2.1 time: 0.154381 Sep 04 01:07:00.819394 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 179/333] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:00 2022] GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215 => generated 1318 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:01.615702 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-2868ee7e-e284-4e88-83e7-e6f9cae954e1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:01.616308 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-2868ee7e-e284-4e88-83e7-e6f9cae954e1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:01.622715 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2868ee7e-e284-4e88-83e7-e6f9cae954e1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:01.623188 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2868ee7e-e284-4e88-83e7-e6f9cae954e1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:01.623619 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2868ee7e-e284-4e88-83e7-e6f9cae954e1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:01.624309 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2868ee7e-e284-4e88-83e7-e6f9cae954e1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:01.624756 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2868ee7e-e284-4e88-83e7-e6f9cae954e1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:01.625207 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2868ee7e-e284-4e88-83e7-e6f9cae954e1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:01.754055 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-2868ee7e-e284-4e88-83e7-e6f9cae954e1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:01.761715 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-2868ee7e-e284-4e88-83e7-e6f9cae954e1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1522 microversion: 2.1 time: 0.149990 Sep 04 01:07:01.762284 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 155/334] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:01 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1522 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:01.844246 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b0171e81-22c5-415f-80e7-77001c136915 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:01.845105 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-b0171e81-22c5-415f-80e7-77001c136915 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: 2808d334-c39c-4a09-88c7-c1711cce4215] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:01.851795 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b0171e81-22c5-415f-80e7-77001c136915 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:01.852628 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b0171e81-22c5-415f-80e7-77001c136915 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:01.853067 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b0171e81-22c5-415f-80e7-77001c136915 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:01.853825 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b0171e81-22c5-415f-80e7-77001c136915 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:01.854320 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b0171e81-22c5-415f-80e7-77001c136915 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:01.854740 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b0171e81-22c5-415f-80e7-77001c136915 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:01.984863 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-b0171e81-22c5-415f-80e7-77001c136915 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:01.993401 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-b0171e81-22c5-415f-80e7-77001c136915 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215" status: 200 len: 1507 microversion: 2.1 time: 0.153177 Sep 04 01:07:01.994138 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 180/335] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:01 2022] GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215 => generated 1507 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:02.785498 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-0d57ecb8-b443-4b89-afec-8faa04bc0f75 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:02.786021 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-0d57ecb8-b443-4b89-afec-8faa04bc0f75 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:02.791996 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0d57ecb8-b443-4b89-afec-8faa04bc0f75 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:02.792477 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0d57ecb8-b443-4b89-afec-8faa04bc0f75 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:02.792913 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0d57ecb8-b443-4b89-afec-8faa04bc0f75 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:02.793624 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0d57ecb8-b443-4b89-afec-8faa04bc0f75 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:02.794075 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0d57ecb8-b443-4b89-afec-8faa04bc0f75 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:02.794561 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0d57ecb8-b443-4b89-afec-8faa04bc0f75 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:02.929884 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-0d57ecb8-b443-4b89-afec-8faa04bc0f75 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:02.937540 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-0d57ecb8-b443-4b89-afec-8faa04bc0f75 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1522 microversion: 2.1 time: 0.156792 Sep 04 01:07:02.938112 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 156/336] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:02 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1522 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:03.960225 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-91b464f5-281d-4db8-a990-967573d1fc91 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:03.960829 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-91b464f5-281d-4db8-a990-967573d1fc91 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:03.967521 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-91b464f5-281d-4db8-a990-967573d1fc91 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:03.968000 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-91b464f5-281d-4db8-a990-967573d1fc91 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:03.968437 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-91b464f5-281d-4db8-a990-967573d1fc91 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:03.969261 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-91b464f5-281d-4db8-a990-967573d1fc91 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:03.969779 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-91b464f5-281d-4db8-a990-967573d1fc91 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:03.970199 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-91b464f5-281d-4db8-a990-967573d1fc91 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:04.105790 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-91b464f5-281d-4db8-a990-967573d1fc91 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:04.114630 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-91b464f5-281d-4db8-a990-967573d1fc91 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1522 microversion: 2.1 time: 0.158799 Sep 04 01:07:04.115191 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 181/337] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:03 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1522 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:05.137780 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-5a91610c-7797-46a8-8168-d6c4bf0728f0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:05.138432 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-5a91610c-7797-46a8-8168-d6c4bf0728f0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:05.145418 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5a91610c-7797-46a8-8168-d6c4bf0728f0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:05.145897 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5a91610c-7797-46a8-8168-d6c4bf0728f0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:05.146326 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5a91610c-7797-46a8-8168-d6c4bf0728f0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:05.147236 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5a91610c-7797-46a8-8168-d6c4bf0728f0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:05.147836 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5a91610c-7797-46a8-8168-d6c4bf0728f0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:05.148110 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5a91610c-7797-46a8-8168-d6c4bf0728f0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:05.332284 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-5a91610c-7797-46a8-8168-d6c4bf0728f0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:05.340484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-5a91610c-7797-46a8-8168-d6c4bf0728f0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1522 microversion: 2.1 time: 0.206557 Sep 04 01:07:05.341042 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 157/338] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:05 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1522 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:05.521940 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-446548a8-3f7b-4d72-afaa-46eed1d21e77 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:05.523003 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-446548a8-3f7b-4d72-afaa-46eed1d21e77 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:05.530324 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-446548a8-3f7b-4d72-afaa-46eed1d21e77 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:05.530754 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-446548a8-3f7b-4d72-afaa-46eed1d21e77 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:05.531260 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-446548a8-3f7b-4d72-afaa-46eed1d21e77 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:05.532038 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-446548a8-3f7b-4d72-afaa-46eed1d21e77 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:05.532426 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-446548a8-3f7b-4d72-afaa-46eed1d21e77 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:05.532848 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-446548a8-3f7b-4d72-afaa-46eed1d21e77 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:05.567817 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-446548a8-3f7b-4d72-afaa-46eed1d21e77 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2/ips" status: 200 len: 145 microversion: 2.1 time: 0.051472 Sep 04 01:07:05.569552 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 182/339] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Sep 4 01:07:05 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2/ips => generated 145 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:07:06.364435 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-8e0312aa-1a0c-4f93-99fb-bf7599853ff4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:06.365206 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-8e0312aa-1a0c-4f93-99fb-bf7599853ff4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:06.371871 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8e0312aa-1a0c-4f93-99fb-bf7599853ff4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:06.372365 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8e0312aa-1a0c-4f93-99fb-bf7599853ff4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:06.372797 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8e0312aa-1a0c-4f93-99fb-bf7599853ff4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:06.373562 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8e0312aa-1a0c-4f93-99fb-bf7599853ff4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:06.374057 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8e0312aa-1a0c-4f93-99fb-bf7599853ff4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:06.374655 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8e0312aa-1a0c-4f93-99fb-bf7599853ff4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:06.516377 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-8e0312aa-1a0c-4f93-99fb-bf7599853ff4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:06.525369 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-8e0312aa-1a0c-4f93-99fb-bf7599853ff4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1522 microversion: 2.1 time: 0.165552 Sep 04 01:07:06.525953 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 158/340] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:06 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1522 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:07.548425 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-a7db84c7-2c7f-4026-b834-0be11618b9fe tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:07.549032 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-a7db84c7-2c7f-4026-b834-0be11618b9fe tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:07.555225 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7db84c7-2c7f-4026-b834-0be11618b9fe tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:07.555708 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7db84c7-2c7f-4026-b834-0be11618b9fe tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:07.556152 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7db84c7-2c7f-4026-b834-0be11618b9fe tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:07.556828 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7db84c7-2c7f-4026-b834-0be11618b9fe tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:07.557310 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7db84c7-2c7f-4026-b834-0be11618b9fe tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:07.557738 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7db84c7-2c7f-4026-b834-0be11618b9fe tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:07.700804 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-a7db84c7-2c7f-4026-b834-0be11618b9fe tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:07.709426 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-a7db84c7-2c7f-4026-b834-0be11618b9fe tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1522 microversion: 2.1 time: 0.164781 Sep 04 01:07:07.710073 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 183/341] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:07 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1522 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:08.732802 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-4bdd984e-8355-4c57-803f-494de97876e8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:08.733474 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-4bdd984e-8355-4c57-803f-494de97876e8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:08.740318 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4bdd984e-8355-4c57-803f-494de97876e8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:08.740791 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4bdd984e-8355-4c57-803f-494de97876e8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:08.741254 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4bdd984e-8355-4c57-803f-494de97876e8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:08.741934 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4bdd984e-8355-4c57-803f-494de97876e8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:08.742378 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4bdd984e-8355-4c57-803f-494de97876e8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:08.742789 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4bdd984e-8355-4c57-803f-494de97876e8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:08.878523 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-4bdd984e-8355-4c57-803f-494de97876e8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:08.886312 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-4bdd984e-8355-4c57-803f-494de97876e8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1522 microversion: 2.1 time: 0.157364 Sep 04 01:07:08.886985 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 159/342] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:08 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1522 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:09.922436 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-ad104c4f-6bea-40a8-9c2d-14c5184218b3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:09.923229 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-ad104c4f-6bea-40a8-9c2d-14c5184218b3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:09.931961 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ad104c4f-6bea-40a8-9c2d-14c5184218b3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:09.932612 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ad104c4f-6bea-40a8-9c2d-14c5184218b3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:09.933182 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ad104c4f-6bea-40a8-9c2d-14c5184218b3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:09.936944 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ad104c4f-6bea-40a8-9c2d-14c5184218b3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:09.947567 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ad104c4f-6bea-40a8-9c2d-14c5184218b3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:09.947567 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ad104c4f-6bea-40a8-9c2d-14c5184218b3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:10.125328 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ad104c4f-6bea-40a8-9c2d-14c5184218b3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:10.136161 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-ad104c4f-6bea-40a8-9c2d-14c5184218b3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1522 microversion: 2.1 time: 0.218029 Sep 04 01:07:10.137472 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 184/343] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:09 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1522 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:07:11.162167 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-b06a9487-0490-48e7-ad1e-4b05256cb226 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:11.164310 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-b06a9487-0490-48e7-ad1e-4b05256cb226 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:11.174765 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b06a9487-0490-48e7-ad1e-4b05256cb226 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:11.174765 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b06a9487-0490-48e7-ad1e-4b05256cb226 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:11.174765 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b06a9487-0490-48e7-ad1e-4b05256cb226 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:11.174765 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b06a9487-0490-48e7-ad1e-4b05256cb226 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:11.175284 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b06a9487-0490-48e7-ad1e-4b05256cb226 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:11.175682 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b06a9487-0490-48e7-ad1e-4b05256cb226 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:11.312116 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-b06a9487-0490-48e7-ad1e-4b05256cb226 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:11.320104 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-b06a9487-0490-48e7-ad1e-4b05256cb226 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1522 microversion: 2.1 time: 0.164062 Sep 04 01:07:11.320668 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 160/344] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:11 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1522 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:11.685271 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-0b86b546-8707-4203-9591-9ae79219b837 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "f9d7184f-5fde-40d3-8812-f912f9298d61"}} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:07:11.687667 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-0b86b546-8707-4203-9591-9ae79219b837 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:11.693865 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0b86b546-8707-4203-9591-9ae79219b837 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:11.697118 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0b86b546-8707-4203-9591-9ae79219b837 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:11.697118 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0b86b546-8707-4203-9591-9ae79219b837 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:11.697118 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0b86b546-8707-4203-9591-9ae79219b837 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:11.697118 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0b86b546-8707-4203-9591-9ae79219b837 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:11.697667 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0b86b546-8707-4203-9591-9ae79219b837 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:12.342508 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-2925dad0-06a9-4246-98b4-725e691abfe7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:12.343286 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-2925dad0-06a9-4246-98b4-725e691abfe7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:12.350096 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2925dad0-06a9-4246-98b4-725e691abfe7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:12.350576 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2925dad0-06a9-4246-98b4-725e691abfe7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:12.351014 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2925dad0-06a9-4246-98b4-725e691abfe7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:12.351744 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2925dad0-06a9-4246-98b4-725e691abfe7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:12.352218 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2925dad0-06a9-4246-98b4-725e691abfe7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:12.352641 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2925dad0-06a9-4246-98b4-725e691abfe7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:12.502200 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-2925dad0-06a9-4246-98b4-725e691abfe7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:12.510951 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-2925dad0-06a9-4246-98b4-725e691abfe7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1522 microversion: 2.1 time: 0.172101 Sep 04 01:07:12.512027 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 161/345] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:12 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1522 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:12.581833 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-0b86b546-8707-4203-9591-9ae79219b837 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "POST /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.900023 Sep 04 01:07:12.582820 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 185/346] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Sep 4 01:07:11 2022] POST /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2/os-volume_attachments => generated 194 bytes in 902 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:07:13.534434 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-e2dc00eb-c121-47d5-b312-623b83c754a1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:13.535219 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-e2dc00eb-c121-47d5-b312-623b83c754a1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:13.543138 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e2dc00eb-c121-47d5-b312-623b83c754a1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:13.543624 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e2dc00eb-c121-47d5-b312-623b83c754a1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:13.544052 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e2dc00eb-c121-47d5-b312-623b83c754a1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:13.544751 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e2dc00eb-c121-47d5-b312-623b83c754a1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:13.545256 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e2dc00eb-c121-47d5-b312-623b83c754a1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:13.545692 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e2dc00eb-c121-47d5-b312-623b83c754a1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:13.689864 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-e2dc00eb-c121-47d5-b312-623b83c754a1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:13.700898 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-e2dc00eb-c121-47d5-b312-623b83c754a1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1522 microversion: 2.1 time: 0.170144 Sep 04 01:07:13.701866 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 162/347] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:13 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1522 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:14.729873 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-ad8016fc-1c5c-471f-8443-2d02d05c40dd tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:14.730493 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-ad8016fc-1c5c-471f-8443-2d02d05c40dd tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:14.737578 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ad8016fc-1c5c-471f-8443-2d02d05c40dd tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:14.738055 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ad8016fc-1c5c-471f-8443-2d02d05c40dd tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:14.738483 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ad8016fc-1c5c-471f-8443-2d02d05c40dd tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:14.739670 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ad8016fc-1c5c-471f-8443-2d02d05c40dd tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:14.740118 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ad8016fc-1c5c-471f-8443-2d02d05c40dd tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:14.740542 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ad8016fc-1c5c-471f-8443-2d02d05c40dd tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:14.894896 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ad8016fc-1c5c-471f-8443-2d02d05c40dd tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:14.902929 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-ad8016fc-1c5c-471f-8443-2d02d05c40dd tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1522 microversion: 2.1 time: 0.177335 Sep 04 01:07:14.903498 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 186/348] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:14 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1522 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:15.926779 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-19352d51-372f-4754-824c-917b42b3bb94 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:15.927545 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-19352d51-372f-4754-824c-917b42b3bb94 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:15.935340 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-19352d51-372f-4754-824c-917b42b3bb94 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:15.935820 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-19352d51-372f-4754-824c-917b42b3bb94 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:15.936252 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-19352d51-372f-4754-824c-917b42b3bb94 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:15.936948 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-19352d51-372f-4754-824c-917b42b3bb94 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:15.937430 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-19352d51-372f-4754-824c-917b42b3bb94 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:15.937854 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-19352d51-372f-4754-824c-917b42b3bb94 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:16.073209 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-19352d51-372f-4754-824c-917b42b3bb94 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:16.080880 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-19352d51-372f-4754-824c-917b42b3bb94 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1522 microversion: 2.1 time: 0.158385 Sep 04 01:07:16.081529 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 163/349] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:15 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1522 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:17.105718 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-a46473d3-12f3-4e56-b0ab-0b767ebd7ffa tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:17.106337 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-a46473d3-12f3-4e56-b0ab-0b767ebd7ffa tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:17.113191 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a46473d3-12f3-4e56-b0ab-0b767ebd7ffa tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:17.113680 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a46473d3-12f3-4e56-b0ab-0b767ebd7ffa tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:17.114106 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a46473d3-12f3-4e56-b0ab-0b767ebd7ffa tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:17.114796 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a46473d3-12f3-4e56-b0ab-0b767ebd7ffa tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:17.115246 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a46473d3-12f3-4e56-b0ab-0b767ebd7ffa tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:17.115665 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a46473d3-12f3-4e56-b0ab-0b767ebd7ffa tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:17.135343 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:17.246333 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-a46473d3-12f3-4e56-b0ab-0b767ebd7ffa tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:17.254825 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-a46473d3-12f3-4e56-b0ab-0b767ebd7ffa tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1522 microversion: 2.1 time: 0.152580 Sep 04 01:07:17.255576 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 187/350] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:17 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1522 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:18.278535 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-999ec07e-7c81-4d27-9277-98f012f5e0a9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:18.279146 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-999ec07e-7c81-4d27-9277-98f012f5e0a9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:18.285743 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-999ec07e-7c81-4d27-9277-98f012f5e0a9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:18.286411 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-999ec07e-7c81-4d27-9277-98f012f5e0a9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:18.286844 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-999ec07e-7c81-4d27-9277-98f012f5e0a9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:18.287532 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-999ec07e-7c81-4d27-9277-98f012f5e0a9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:18.287984 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-999ec07e-7c81-4d27-9277-98f012f5e0a9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:18.288408 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-999ec07e-7c81-4d27-9277-98f012f5e0a9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:18.290988 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:18.424891 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-999ec07e-7c81-4d27-9277-98f012f5e0a9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:18.433508 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-999ec07e-7c81-4d27-9277-98f012f5e0a9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1522 microversion: 2.1 time: 0.158747 Sep 04 01:07:18.434072 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 164/351] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:18 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1522 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:19.458218 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-020b8d40-3dc0-4471-8674-b9cda2c4ab7a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:19.458836 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-020b8d40-3dc0-4471-8674-b9cda2c4ab7a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:19.461354 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_api:SELECT=44 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:19.466401 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-020b8d40-3dc0-4471-8674-b9cda2c4ab7a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:19.467087 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-020b8d40-3dc0-4471-8674-b9cda2c4ab7a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:19.467524 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-020b8d40-3dc0-4471-8674-b9cda2c4ab7a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:19.468354 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-020b8d40-3dc0-4471-8674-b9cda2c4ab7a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:19.468814 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-020b8d40-3dc0-4471-8674-b9cda2c4ab7a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:19.469304 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-020b8d40-3dc0-4471-8674-b9cda2c4ab7a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:19.561694 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-ad1caa58-fede-4627-9bd3-57e00bf656ef tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:19.562859 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-ad1caa58-fede-4627-9bd3-57e00bf656ef tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:19.570537 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ad1caa58-fede-4627-9bd3-57e00bf656ef tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:19.571041 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ad1caa58-fede-4627-9bd3-57e00bf656ef tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:19.571464 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ad1caa58-fede-4627-9bd3-57e00bf656ef tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:19.572168 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ad1caa58-fede-4627-9bd3-57e00bf656ef tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:19.572619 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ad1caa58-fede-4627-9bd3-57e00bf656ef tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:19.573045 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ad1caa58-fede-4627-9bd3-57e00bf656ef tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:19.598653 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-ad1caa58-fede-4627-9bd3-57e00bf656ef tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138/ips" status: 200 len: 151 microversion: 2.1 time: 0.040569 Sep 04 01:07:19.599256 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 165/352] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Sep 4 01:07:19 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138/ips => generated 151 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:07:19.609186 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-020b8d40-3dc0-4471-8674-b9cda2c4ab7a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:19.616828 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-020b8d40-3dc0-4471-8674-b9cda2c4ab7a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1522 microversion: 2.1 time: 0.163120 Sep 04 01:07:19.617596 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 188/353] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:19 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1522 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:20.640254 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-6e2b463c-1b56-4e03-a71c-3c5260e38bec tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:20.640864 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-6e2b463c-1b56-4e03-a71c-3c5260e38bec tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:20.647653 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6e2b463c-1b56-4e03-a71c-3c5260e38bec tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:20.648138 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6e2b463c-1b56-4e03-a71c-3c5260e38bec tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:20.655932 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6e2b463c-1b56-4e03-a71c-3c5260e38bec tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:20.655932 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6e2b463c-1b56-4e03-a71c-3c5260e38bec tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:20.655932 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6e2b463c-1b56-4e03-a71c-3c5260e38bec tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:20.655932 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6e2b463c-1b56-4e03-a71c-3c5260e38bec tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:20.803844 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-6e2b463c-1b56-4e03-a71c-3c5260e38bec tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:20.811495 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-6e2b463c-1b56-4e03-a71c-3c5260e38bec tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1502 microversion: 2.1 time: 0.174953 Sep 04 01:07:20.812052 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 166/354] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:20 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1502 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:20.834502 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-c4809ada-fb38-44ed-afa2-f8da888fc131 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "123b82be-2e16-43b0-9ab1-62c600b33a9c"}} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:07:20.836886 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-c4809ada-fb38-44ed-afa2-f8da888fc131 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:20.843439 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c4809ada-fb38-44ed-afa2-f8da888fc131 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:20.843919 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c4809ada-fb38-44ed-afa2-f8da888fc131 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:20.844360 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c4809ada-fb38-44ed-afa2-f8da888fc131 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:20.845156 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c4809ada-fb38-44ed-afa2-f8da888fc131 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:20.845613 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c4809ada-fb38-44ed-afa2-f8da888fc131 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:20.846036 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c4809ada-fb38-44ed-afa2-f8da888fc131 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:20.872908 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.wsgi [None req-c4809ada-fb38-44ed-afa2-f8da888fc131 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] HTTP exception thrown: Cannot 'attach_volume' instance 2f7a7e72-1f16-4ab4-b826-d376d39be4c2 while it is in vm_state rescued Sep 04 01:07:20.874910 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-c4809ada-fb38-44ed-afa2-f8da888fc131 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Request method failure captured: Sep 04 01:07:20.874910 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: request: POST /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2/os-volume_attachments HTTP/1.1 Sep 04 01:07:20.874910 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept: application/json Sep 04 01:07:20.874910 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept-Encoding: identity Sep 04 01:07:20.874910 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Connection: close Sep 04 01:07:20.874910 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Length: 74 Sep 04 01:07:20.874910 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Type: application/json Sep 04 01:07:20.874910 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Host: 10.222.0.42 Sep 04 01:07:20.874910 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Openstack-System-Scope: None Sep 04 01:07:20.874910 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: User-Agent: python-urllib3/1.26.12 Sep 04 01:07:20.874910 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Auth-Token: gAAAAABjE_nMCq4zD8BmB6mF8h26w0y_IFei-x7tMbI5vcKfWFaeAi75nOGGI72_ZHVfHvHOCK5qrlGll_NneJYvZkWgGCkELi_QIwkIIEFfz91_ouiYPU3bkvjmjKoIsPt_9g8qfiCyLaERnndIFBkeIUe0g0NvmODNU753JT7NKZpvzaTLcsI Sep 04 01:07:20.874910 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Id: None Sep 04 01:07:20.874910 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Name: None Sep 04 01:07:20.874910 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Identity-Status: Confirmed Sep 04 01:07:20.874910 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Is-Admin-Project: True Sep 04 01:07:20.874910 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Id: default Sep 04 01:07:20.874910 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Name: Default Sep 04 01:07:20.874910 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Id: db72b63281384bd883b04d05bcc35677 Sep 04 01:07:20.874910 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1367456352 Sep 04 01:07:20.874910 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Role: reader,member Sep 04 01:07:20.874910 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Roles: reader,member Sep 04 01:07:20.876390 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/db72b63281384bd883b04d05bcc35677"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_db72b63281384bd883b04d05bcc35677", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/db72b63281384bd883b04d05bcc35677"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/db72b63281384bd883b04d05bcc35677"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:07:20.876390 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1367456352 Sep 04 01:07:20.876390 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Id: db72b63281384bd883b04d05bcc35677 Sep 04 01:07:20.876390 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1367456352 Sep 04 01:07:20.876390 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User: tempest-ServerRescueNegativeTestJSON-1367456352-project Sep 04 01:07:20.876390 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Id: default Sep 04 01:07:20.876390 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Name: Default Sep 04 01:07:20.876390 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Id: bd13df3a2d0e4182bbd86dab7035c371 Sep 04 01:07:20.876390 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1367456352-project Sep 04 01:07:20.876390 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Sep 04 01:07:20.876390 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: {"volumeAttachment": {"volumeId": "123b82be-2e16-43b0-9ab1-62c600b33a9c"}} Sep 04 01:07:20.876390 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: method: > Sep 04 01:07:20.877455 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: exception: Cannot 'attach_volume' instance 2f7a7e72-1f16-4ab4-b826-d376d39be4c2 while it is in vm_state rescued Sep 04 01:07:20.877455 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: action_args: {'server_id': '2f7a7e72-1f16-4ab4-b826-d376d39be4c2', 'body': {'volumeAttachment': {'volumeId': '123b82be-2e16-43b0-9ab1-62c600b33a9c'}}} Sep 04 01:07:20.877455 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:07:20.877455 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:07:20.877455 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Sep 04 01:07:20.877455 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Sep 04 01:07:20.877455 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 04 01:07:20.877455 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 04 01:07:20.877455 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Sep 04 01:07:20.877455 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 04 01:07:20.877455 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 2f7a7e72-1f16-4ab4-b826-d376d39be4c2 in vm_state rescued. Cannot attach_volume while the instance is in this state. Sep 04 01:07:20.877455 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:07:20.877455 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:07:20.877455 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:07:20.877455 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:07:20.877455 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:07:20.877455 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:07:20.877455 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:07:20.877455 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:07:20.878740 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:07:20.878740 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:07:20.878740 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 04 01:07:20.878740 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 04 01:07:20.878740 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 04 01:07:20.878740 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 04 01:07:20.878740 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 04 01:07:20.878740 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 04 01:07:20.878740 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Sep 04 01:07:20.878740 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Sep 04 01:07:20.878740 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 04 01:07:20.878740 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 04 01:07:20.878740 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance 2f7a7e72-1f16-4ab4-b826-d376d39be4c2 while it is in vm_state rescued Sep 04 01:07:20.878740 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:07:20.878740 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:07:20.878740 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:07:20.878740 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:07:20.878740 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:07:20.878740 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:07:20.880049 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:07:20.880049 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:07:20.880049 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance 2f7a7e72-1f16-4ab4-b826-d376d39be4c2 while it is in vm_state rescued Sep 04 01:07:20.880049 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:07:20.881151 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-c4809ada-fb38-44ed-afa2-f8da888fc131 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Returning 409 to user: Cannot 'attach_volume' instance 2f7a7e72-1f16-4ab4-b826-d376d39be4c2 while it is in vm_state rescued {{(pid=121036) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:07:20.882055 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-c4809ada-fb38-44ed-afa2-f8da888fc131 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "POST /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.052359 Sep 04 01:07:20.884110 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 189/355] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Sep 4 01:07:20 2022] POST /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2/os-volume_attachments => generated 152 bytes in 55 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Sep 04 01:07:20.892728 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-81699e54-f525-4a81-926f-b24414d16a9b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:07:20.893479 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-81699e54-f525-4a81-926f-b24414d16a9b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:20.901413 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-81699e54-f525-4a81-926f-b24414d16a9b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:20.902014 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-81699e54-f525-4a81-926f-b24414d16a9b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:20.902782 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-81699e54-f525-4a81-926f-b24414d16a9b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:20.903671 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-81699e54-f525-4a81-926f-b24414d16a9b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:20.907227 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-81699e54-f525-4a81-926f-b24414d16a9b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:20.907227 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-81699e54-f525-4a81-926f-b24414d16a9b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:21.164290 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-81699e54-f525-4a81-926f-b24414d16a9b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "POST /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2/action" status: 202 len: 0 microversion: 2.1 time: 0.270739 Sep 04 01:07:21.164290 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 167/356] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Sep 4 01:07:20 2022] POST /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2/action => generated 0 bytes in 272 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 04 01:07:21.170914 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-eae50b8d-3c47-4dac-824e-3bdf09387bc9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:21.171927 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-eae50b8d-3c47-4dac-824e-3bdf09387bc9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:21.180034 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-eae50b8d-3c47-4dac-824e-3bdf09387bc9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:21.180950 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-eae50b8d-3c47-4dac-824e-3bdf09387bc9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:21.181418 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-eae50b8d-3c47-4dac-824e-3bdf09387bc9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:21.182275 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-eae50b8d-3c47-4dac-824e-3bdf09387bc9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:21.183166 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-eae50b8d-3c47-4dac-824e-3bdf09387bc9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:21.183964 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-eae50b8d-3c47-4dac-824e-3bdf09387bc9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:21.322733 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-eae50b8d-3c47-4dac-824e-3bdf09387bc9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:21.331183 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-eae50b8d-3c47-4dac-824e-3bdf09387bc9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1510 microversion: 2.1 time: 0.165725 Sep 04 01:07:21.331755 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 190/357] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:21 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1510 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:21.481916 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-f15a9173-24a4-461a-acbc-55f9876dfcbe tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "c8868d57-3f95-40ba-8bb5-0b47ba862842"}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:07:21.484435 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-f15a9173-24a4-461a-acbc-55f9876dfcbe tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:21.491599 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f15a9173-24a4-461a-acbc-55f9876dfcbe tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:21.492083 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f15a9173-24a4-461a-acbc-55f9876dfcbe tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:21.492681 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f15a9173-24a4-461a-acbc-55f9876dfcbe tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:21.493556 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f15a9173-24a4-461a-acbc-55f9876dfcbe tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:21.494153 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f15a9173-24a4-461a-acbc-55f9876dfcbe tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:21.494736 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f15a9173-24a4-461a-acbc-55f9876dfcbe tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:22.354765 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-6db1d83a-30c8-46b7-98f3-6e2a7a54179b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:22.355406 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-6db1d83a-30c8-46b7-98f3-6e2a7a54179b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:22.362027 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6db1d83a-30c8-46b7-98f3-6e2a7a54179b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:22.362631 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6db1d83a-30c8-46b7-98f3-6e2a7a54179b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:22.363065 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6db1d83a-30c8-46b7-98f3-6e2a7a54179b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:22.364274 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6db1d83a-30c8-46b7-98f3-6e2a7a54179b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:22.364724 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6db1d83a-30c8-46b7-98f3-6e2a7a54179b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:22.365178 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6db1d83a-30c8-46b7-98f3-6e2a7a54179b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:22.471911 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-f15a9173-24a4-461a-acbc-55f9876dfcbe tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "POST /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.994203 Sep 04 01:07:22.473093 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 168/358] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Sep 4 01:07:21 2022] POST /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138/os-volume_attachments => generated 194 bytes in 996 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:07:22.490903 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-57761032-a04a-483c-9fb3-53733b783169 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:22.493391 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-57761032-a04a-483c-9fb3-53733b783169 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:22.501297 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-57761032-a04a-483c-9fb3-53733b783169 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:22.502438 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-57761032-a04a-483c-9fb3-53733b783169 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:22.503152 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-57761032-a04a-483c-9fb3-53733b783169 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:22.504617 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-57761032-a04a-483c-9fb3-53733b783169 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:22.505630 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-57761032-a04a-483c-9fb3-53733b783169 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:22.506255 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-57761032-a04a-483c-9fb3-53733b783169 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:22.513055 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-6db1d83a-30c8-46b7-98f3-6e2a7a54179b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:22.521451 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-6db1d83a-30c8-46b7-98f3-6e2a7a54179b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1510 microversion: 2.1 time: 0.170430 Sep 04 01:07:22.522375 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 191/359] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:22 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1510 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:07:22.553359 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-57761032-a04a-483c-9fb3-53733b783169 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3/ips" status: 200 len: 150 microversion: 2.20 time: 0.078850 Sep 04 01:07:22.553996 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 169/360] 10.222.0.42 () {60 vars in 1289 bytes} [Sun Sep 4 01:07:22 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3/ips => generated 150 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 04 01:07:23.550282 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-61c9a8dc-bf4b-4b2e-b6f1-7ffa7f4c3334 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:23.550882 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-61c9a8dc-bf4b-4b2e-b6f1-7ffa7f4c3334 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:23.557349 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-61c9a8dc-bf4b-4b2e-b6f1-7ffa7f4c3334 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:23.557826 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-61c9a8dc-bf4b-4b2e-b6f1-7ffa7f4c3334 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:23.558255 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-61c9a8dc-bf4b-4b2e-b6f1-7ffa7f4c3334 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:23.558930 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-61c9a8dc-bf4b-4b2e-b6f1-7ffa7f4c3334 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:23.559374 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-61c9a8dc-bf4b-4b2e-b6f1-7ffa7f4c3334 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:23.559794 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-61c9a8dc-bf4b-4b2e-b6f1-7ffa7f4c3334 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:23.708488 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-61c9a8dc-bf4b-4b2e-b6f1-7ffa7f4c3334 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:23.717010 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-61c9a8dc-bf4b-4b2e-b6f1-7ffa7f4c3334 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1510 microversion: 2.1 time: 0.170507 Sep 04 01:07:23.717627 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 192/361] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:23 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1510 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:24.740793 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-dd278a3c-0441-44f0-82c3-f68b3b0da163 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:24.741428 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-dd278a3c-0441-44f0-82c3-f68b3b0da163 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:24.748470 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-dd278a3c-0441-44f0-82c3-f68b3b0da163 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:24.749084 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-dd278a3c-0441-44f0-82c3-f68b3b0da163 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:24.749538 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-dd278a3c-0441-44f0-82c3-f68b3b0da163 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:24.750232 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-dd278a3c-0441-44f0-82c3-f68b3b0da163 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:24.750687 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-dd278a3c-0441-44f0-82c3-f68b3b0da163 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:24.751109 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-dd278a3c-0441-44f0-82c3-f68b3b0da163 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:24.957219 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-dd278a3c-0441-44f0-82c3-f68b3b0da163 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:24.965024 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-dd278a3c-0441-44f0-82c3-f68b3b0da163 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1510 microversion: 2.1 time: 0.228676 Sep 04 01:07:24.965633 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 170/362] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:24 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1510 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:25.267411 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-fc3a87d3-1ca8-4441-a66d-21ccf001a0d2 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "19c48c1c-9d94-45af-a46a-d4af0c2d845c", "device": "/dev/sdb"}} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:07:25.270325 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-fc3a87d3-1ca8-4441-a66d-21ccf001a0d2 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: 2808d334-c39c-4a09-88c7-c1711cce4215] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:25.277868 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fc3a87d3-1ca8-4441-a66d-21ccf001a0d2 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:25.278341 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fc3a87d3-1ca8-4441-a66d-21ccf001a0d2 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:25.278776 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fc3a87d3-1ca8-4441-a66d-21ccf001a0d2 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:25.279473 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fc3a87d3-1ca8-4441-a66d-21ccf001a0d2 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:25.279921 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fc3a87d3-1ca8-4441-a66d-21ccf001a0d2 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:25.280342 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fc3a87d3-1ca8-4441-a66d-21ccf001a0d2 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:25.876443 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-746b9370-316d-450b-ae78-836fe5ada2fd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:07:25.877030 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-746b9370-316d-450b-ae78-836fe5ada2fd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:25.883956 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-746b9370-316d-450b-ae78-836fe5ada2fd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:25.884435 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-746b9370-316d-450b-ae78-836fe5ada2fd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:25.884861 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-746b9370-316d-450b-ae78-836fe5ada2fd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:25.885566 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-746b9370-316d-450b-ae78-836fe5ada2fd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:25.886008 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-746b9370-316d-450b-ae78-836fe5ada2fd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:25.886427 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-746b9370-316d-450b-ae78-836fe5ada2fd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:25.909980 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.objects.instance [None req-746b9370-316d-450b-ae78-836fe5ada2fd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lazy-loading 'flavor' on Instance uuid 49be5a0b-f748-486e-a546-c07a8e8451a3 {{(pid=121037) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:07:26.473774 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-746b9370-316d-450b-ae78-836fe5ada2fd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "POST /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3/action" status: 202 len: 0 microversion: 2.20 time: 0.600662 Sep 04 01:07:26.475057 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 171/363] 10.222.0.42 () {62 vars in 1319 bytes} [Sun Sep 4 01:07:25 2022] POST /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3/action => generated 0 bytes in 603 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Sep 04 01:07:26.480181 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-af99e78e-e806-4cc9-904c-e2bf897b87d1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:26.480878 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-af99e78e-e806-4cc9-904c-e2bf897b87d1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:26.487557 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-af99e78e-e806-4cc9-904c-e2bf897b87d1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:26.487980 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-af99e78e-e806-4cc9-904c-e2bf897b87d1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:26.488414 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-af99e78e-e806-4cc9-904c-e2bf897b87d1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:26.489129 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-af99e78e-e806-4cc9-904c-e2bf897b87d1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:26.489586 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-af99e78e-e806-4cc9-904c-e2bf897b87d1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:26.490630 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-af99e78e-e806-4cc9-904c-e2bf897b87d1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:26.637037 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-af99e78e-e806-4cc9-904c-e2bf897b87d1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:26.646607 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-af99e78e-e806-4cc9-904c-e2bf897b87d1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1516 microversion: 2.1 time: 0.169856 Sep 04 01:07:26.647261 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 172/364] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:26 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1516 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:26.652077 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-4270ba08-5491-4140-b659-e0ce907e8dce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:26.652688 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-4270ba08-5491-4140-b659-e0ce907e8dce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:26.665409 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4270ba08-5491-4140-b659-e0ce907e8dce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:26.665409 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4270ba08-5491-4140-b659-e0ce907e8dce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:26.665409 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4270ba08-5491-4140-b659-e0ce907e8dce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:26.665409 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4270ba08-5491-4140-b659-e0ce907e8dce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:26.665875 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4270ba08-5491-4140-b659-e0ce907e8dce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:26.665875 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4270ba08-5491-4140-b659-e0ce907e8dce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:26.813580 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-4270ba08-5491-4140-b659-e0ce907e8dce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:26.827468 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-4270ba08-5491-4140-b659-e0ce907e8dce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:26.829935 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-4270ba08-5491-4140-b659-e0ce907e8dce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:26.831219 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-4270ba08-5491-4140-b659-e0ce907e8dce tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1721 microversion: 2.20 time: 0.182838 Sep 04 01:07:26.832217 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 173/365] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:07:26 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1721 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:07:26.948738 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-a06bf29f-0388-4f9e-8aac-525e86026037 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:07:26.955182 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a06bf29f-0388-4f9e-8aac-525e86026037 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:26.955663 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a06bf29f-0388-4f9e-8aac-525e86026037 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:26.956099 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a06bf29f-0388-4f9e-8aac-525e86026037 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:27.030635 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-a06bf29f-0388-4f9e-8aac-525e86026037 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Going to try to stop instance {{(pid=121037) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Sep 04 01:07:27.054174 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-fc3a87d3-1ca8-4441-a66d-21ccf001a0d2 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "POST /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.790502 Sep 04 01:07:27.062060 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 193/366] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Sep 4 01:07:25 2022] POST /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215/os-volume_attachments => generated 194 bytes in 1799 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:07:27.237536 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-a06bf29f-0388-4f9e-8aac-525e86026037 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "POST /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2/action" status: 202 len: 0 microversion: 2.1 time: 0.292287 Sep 04 01:07:27.238144 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 174/367] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Sep 4 01:07:26 2022] POST /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2/action => generated 0 bytes in 294 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 04 01:07:27.247933 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-5fa8d0ff-8d5c-4a53-a26c-f3d89b448e97 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:27.248577 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-5fa8d0ff-8d5c-4a53-a26c-f3d89b448e97 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:27.254847 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5fa8d0ff-8d5c-4a53-a26c-f3d89b448e97 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:27.255286 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5fa8d0ff-8d5c-4a53-a26c-f3d89b448e97 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:27.255739 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5fa8d0ff-8d5c-4a53-a26c-f3d89b448e97 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:27.256417 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5fa8d0ff-8d5c-4a53-a26c-f3d89b448e97 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:27.256867 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5fa8d0ff-8d5c-4a53-a26c-f3d89b448e97 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:27.257353 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5fa8d0ff-8d5c-4a53-a26c-f3d89b448e97 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:27.414836 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-5fa8d0ff-8d5c-4a53-a26c-f3d89b448e97 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:27.414836 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-5fa8d0ff-8d5c-4a53-a26c-f3d89b448e97 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1720 microversion: 2.1 time: 0.171389 Sep 04 01:07:27.415351 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 194/368] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:27 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1720 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:27.789165 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-cb4fe39a-ebd8-43f5-bf5d-c58fdd097233 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1230339745", "imageRef": "a11315c3-ce81-478b-91e4-5353f189dfa7", "flavorRef": "452", "networks": [{"uuid": "7e04f73d-bad8-4860-94c2-08b2d4e17fa9"}]}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:07:27.852321 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.network.neutron [None req-cb4fe39a-ebd8-43f5-bf5d-c58fdd097233 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] validate_networks() for [('7e04f73d-bad8-4860-94c2-08b2d4e17fa9', None, None, None, None, None)] {{(pid=121037) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 04 01:07:27.858326 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-ff1dcb79-e637-44a0-a662-8d5ad70af195 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:27.858984 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-ff1dcb79-e637-44a0-a662-8d5ad70af195 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:27.865800 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ff1dcb79-e637-44a0-a662-8d5ad70af195 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:27.866384 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ff1dcb79-e637-44a0-a662-8d5ad70af195 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:27.866821 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ff1dcb79-e637-44a0-a662-8d5ad70af195 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:27.867513 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ff1dcb79-e637-44a0-a662-8d5ad70af195 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:27.867971 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ff1dcb79-e637-44a0-a662-8d5ad70af195 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:27.868396 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ff1dcb79-e637-44a0-a662-8d5ad70af195 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:28.011293 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ff1dcb79-e637-44a0-a662-8d5ad70af195 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:28.020238 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ff1dcb79-e637-44a0-a662-8d5ad70af195 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:28.022175 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ff1dcb79-e637-44a0-a662-8d5ad70af195 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:28.023524 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-ff1dcb79-e637-44a0-a662-8d5ad70af195 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1721 microversion: 2.20 time: 0.168977 Sep 04 01:07:28.038185 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 195/369] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:07:27 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1721 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:07:28.231592 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-cb4fe39a-ebd8-43f5-bf5d-c58fdd097233 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Flavor limits 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 04 01:07:28.232053 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-cb4fe39a-ebd8-43f5-bf5d-c58fdd097233 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Image limits 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 04 01:07:28.232508 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-cb4fe39a-ebd8-43f5-bf5d-c58fdd097233 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Flavor pref 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 04 01:07:28.232917 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-cb4fe39a-ebd8-43f5-bf5d-c58fdd097233 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Image pref 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 04 01:07:28.233368 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-cb4fe39a-ebd8-43f5-bf5d-c58fdd097233 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 04 01:07:28.243332 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-cb4fe39a-ebd8-43f5-bf5d-c58fdd097233 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:28.243814 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-cb4fe39a-ebd8-43f5-bf5d-c58fdd097233 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:28.244248 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-cb4fe39a-ebd8-43f5-bf5d-c58fdd097233 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:28.245010 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-cb4fe39a-ebd8-43f5-bf5d-c58fdd097233 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:28.245491 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-cb4fe39a-ebd8-43f5-bf5d-c58fdd097233 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:28.245915 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-cb4fe39a-ebd8-43f5-bf5d-c58fdd097233 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:28.263142 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.quota [None req-cb4fe39a-ebd8-43f5-bf5d-c58fdd097233 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Getting quotas for project bcf6c738f8d842f99d2eec3b896c5b47. Resources: {'ram', 'instances', 'cores'} {{(pid=121037) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 04 01:07:28.267184 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.quota [None req-cb4fe39a-ebd8-43f5-bf5d-c58fdd097233 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Getting quotas for user a6cf41aaddce4a86825bff00477793f9 and project bcf6c738f8d842f99d2eec3b896c5b47. Resources: {'ram', 'instances', 'cores'} {{(pid=121037) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 04 01:07:28.278176 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-cb4fe39a-ebd8-43f5-bf5d-c58fdd097233 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Going to run 1 instances... {{(pid=121037) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 04 01:07:28.282687 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-cb4fe39a-ebd8-43f5-bf5d-c58fdd097233 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a11315c3-ce81-478b-91e4-5353f189dfa7',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121037) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 04 01:07:28.433513 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-cb4fe39a-ebd8-43f5-bf5d-c58fdd097233 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.648485 Sep 04 01:07:28.438891 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-6880c410-f213-4beb-990f-249ccc8d58ae tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:28.440289 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-6880c410-f213-4beb-990f-249ccc8d58ae tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:28.440783 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 175/370] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 4 01:07:27 2022] POST /compute/v2.1/servers => generated 380 bytes in 657 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 04 01:07:28.450295 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-32d484e1-8f34-47fe-a0e7-e0348ce369dc tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:28.451255 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-32d484e1-8f34-47fe-a0e7-e0348ce369dc tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:28.454399 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6880c410-f213-4beb-990f-249ccc8d58ae tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:28.455193 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6880c410-f213-4beb-990f-249ccc8d58ae tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:28.456074 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6880c410-f213-4beb-990f-249ccc8d58ae tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:28.457114 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6880c410-f213-4beb-990f-249ccc8d58ae tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:28.457710 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6880c410-f213-4beb-990f-249ccc8d58ae tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:28.458376 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6880c410-f213-4beb-990f-249ccc8d58ae tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:28.522873 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-32d484e1-8f34-47fe-a0e7-e0348ce369dc tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:28.530632 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-32d484e1-8f34-47fe-a0e7-e0348ce369dc tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5" status: 200 len: 1214 microversion: 2.1 time: 0.088654 Sep 04 01:07:28.540428 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 176/371] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:28 2022] GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 => generated 1214 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:28.540428 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-912aafd7-0266-4a07-a1e7-33748f3aee52 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Action: 'create', calling method: .version_select at 0x7fc10c6c6700>, body: {"keypair": {"name": "tempest-keypair-1002954839", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBK7UGS6+djCGzWWIUlTtr6MYwp+/pG5hklEREP1E/HwgkZKUMYYqE9G48A+tYlvPd7EUgIJQU9eL7lwmGcvFM7I4NfyOLYkmHft258QYRbPB0gxIW7XrBpZKZpGWJ3KQzg=="}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:07:28.553538 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.quota [None req-912aafd7-0266-4a07-a1e7-33748f3aee52 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Getting quotas for project db72b63281384bd883b04d05bcc35677. Resources: {'key_pairs'} {{(pid=121037) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 04 01:07:28.557716 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.quota [None req-912aafd7-0266-4a07-a1e7-33748f3aee52 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Getting quotas for user bd13df3a2d0e4182bbd86dab7035c371 and project db72b63281384bd883b04d05bcc35677. Resources: {'key_pairs'} {{(pid=121037) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 04 01:07:28.635913 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-6880c410-f213-4beb-990f-249ccc8d58ae tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:28.644333 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-6880c410-f213-4beb-990f-249ccc8d58ae tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1720 microversion: 2.1 time: 0.209239 Sep 04 01:07:28.644886 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 196/372] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:28 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1720 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:07:28.664724 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-912aafd7-0266-4a07-a1e7-33748f3aee52 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.132180 Sep 04 01:07:28.665363 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 177/373] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Sep 4 01:07:28 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:07:29.064990 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-e5cb952e-bb5e-414b-9d26-ff7c4d99c2fc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:29.066092 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-e5cb952e-bb5e-414b-9d26-ff7c4d99c2fc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:29.075071 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e5cb952e-bb5e-414b-9d26-ff7c4d99c2fc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:29.076538 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e5cb952e-bb5e-414b-9d26-ff7c4d99c2fc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:29.076538 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e5cb952e-bb5e-414b-9d26-ff7c4d99c2fc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:29.076699 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e5cb952e-bb5e-414b-9d26-ff7c4d99c2fc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:29.077106 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e5cb952e-bb5e-414b-9d26-ff7c4d99c2fc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:29.077624 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e5cb952e-bb5e-414b-9d26-ff7c4d99c2fc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:29.255585 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e5cb952e-bb5e-414b-9d26-ff7c4d99c2fc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:29.263770 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e5cb952e-bb5e-414b-9d26-ff7c4d99c2fc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:29.265326 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e5cb952e-bb5e-414b-9d26-ff7c4d99c2fc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:29.266486 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-e5cb952e-bb5e-414b-9d26-ff7c4d99c2fc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1721 microversion: 2.20 time: 0.205431 Sep 04 01:07:29.267223 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 197/374] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:07:29 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1721 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:07:29.557688 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-b1604a1c-7a3a-4b7a-bb88-dc1658f2138f tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:29.558298 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-b1604a1c-7a3a-4b7a-bb88-dc1658f2138f tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:29.560230 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_api:SELECT=72,nova_api:INSERT=4 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:29.611186 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-b1604a1c-7a3a-4b7a-bb88-dc1658f2138f tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:29.618264 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-b1604a1c-7a3a-4b7a-bb88-dc1658f2138f tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5" status: 200 len: 1214 microversion: 2.1 time: 0.065155 Sep 04 01:07:29.618823 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 178/375] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:29 2022] GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 => generated 1214 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:29.668555 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-980b2f3c-6c6d-4119-a9b3-42a379ba7633 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:29.669132 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-980b2f3c-6c6d-4119-a9b3-42a379ba7633 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:29.675436 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-980b2f3c-6c6d-4119-a9b3-42a379ba7633 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:29.675893 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-980b2f3c-6c6d-4119-a9b3-42a379ba7633 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:29.676329 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-980b2f3c-6c6d-4119-a9b3-42a379ba7633 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:29.677106 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-980b2f3c-6c6d-4119-a9b3-42a379ba7633 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:29.677550 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-980b2f3c-6c6d-4119-a9b3-42a379ba7633 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:29.677979 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-980b2f3c-6c6d-4119-a9b3-42a379ba7633 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:29.810742 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-980b2f3c-6c6d-4119-a9b3-42a379ba7633 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:29.820225 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-980b2f3c-6c6d-4119-a9b3-42a379ba7633 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1720 microversion: 2.1 time: 0.155592 Sep 04 01:07:29.821375 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 198/376] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:29 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1720 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:30.292630 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-31ecd05b-8524-45a4-bab9-a56ae6524e00 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:30.293355 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-31ecd05b-8524-45a4-bab9-a56ae6524e00 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:30.300029 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-31ecd05b-8524-45a4-bab9-a56ae6524e00 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:30.300571 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-31ecd05b-8524-45a4-bab9-a56ae6524e00 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:30.301068 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-31ecd05b-8524-45a4-bab9-a56ae6524e00 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:30.301854 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-31ecd05b-8524-45a4-bab9-a56ae6524e00 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:30.302443 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-31ecd05b-8524-45a4-bab9-a56ae6524e00 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:30.303004 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-31ecd05b-8524-45a4-bab9-a56ae6524e00 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:30.643290 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-53df7311-f53a-4091-b48f-40c4cfe61b5c tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:30.643905 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-53df7311-f53a-4091-b48f-40c4cfe61b5c tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:30.653981 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-53df7311-f53a-4091-b48f-40c4cfe61b5c tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:30.660223 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-53df7311-f53a-4091-b48f-40c4cfe61b5c tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:30.660694 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-53df7311-f53a-4091-b48f-40c4cfe61b5c tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:30.661452 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-53df7311-f53a-4091-b48f-40c4cfe61b5c tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:30.662144 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-53df7311-f53a-4091-b48f-40c4cfe61b5c tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:30.662768 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-53df7311-f53a-4091-b48f-40c4cfe61b5c tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:30.726684 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-31ecd05b-8524-45a4-bab9-a56ae6524e00 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:30.729026 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=90,nova_cell1:UPDATE=6,nova_cell1:INSERT=4 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:30.733634 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-53df7311-f53a-4091-b48f-40c4cfe61b5c tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:30.736932 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-31ecd05b-8524-45a4-bab9-a56ae6524e00 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:30.738410 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-31ecd05b-8524-45a4-bab9-a56ae6524e00 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:30.740161 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-31ecd05b-8524-45a4-bab9-a56ae6524e00 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1721 microversion: 2.20 time: 0.451058 Sep 04 01:07:30.740929 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 179/377] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:07:30 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1721 bytes in 453 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:07:30.748475 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-53df7311-f53a-4091-b48f-40c4cfe61b5c tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5" status: 200 len: 1266 microversion: 2.1 time: 0.102971 Sep 04 01:07:30.748475 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 199/378] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:30 2022] GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 => generated 1266 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:07:30.843570 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-ea3c8cc8-0252-4957-9a25-99a32f9a5334 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:30.844436 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-ea3c8cc8-0252-4957-9a25-99a32f9a5334 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:30.852366 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea3c8cc8-0252-4957-9a25-99a32f9a5334 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:30.852984 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea3c8cc8-0252-4957-9a25-99a32f9a5334 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:30.853551 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea3c8cc8-0252-4957-9a25-99a32f9a5334 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:30.854373 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea3c8cc8-0252-4957-9a25-99a32f9a5334 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:30.854831 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea3c8cc8-0252-4957-9a25-99a32f9a5334 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:30.859510 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea3c8cc8-0252-4957-9a25-99a32f9a5334 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:31.027927 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-ea3c8cc8-0252-4957-9a25-99a32f9a5334 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:31.036082 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-ea3c8cc8-0252-4957-9a25-99a32f9a5334 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1720 microversion: 2.1 time: 0.196297 Sep 04 01:07:31.037052 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 180/379] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:30 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1720 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:31.730819 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-051114d3-60e5-48bd-9eca-32648aab2ca0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1028468859", "imageRef": "a11315c3-ce81-478b-91e4-5353f189dfa7", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "b16f22e1-6fb1-440b-98eb-845123e960e6"}], "security_groups": [{"name": "tempest-securitygroup--938530870"}], "key_name": "tempest-keypair-1002954839", "user_data": "}} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:07:31.765104 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-ae594fd6-d4d4-43b8-8b93-ee3135f5f4f6 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:31.765725 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-ae594fd6-d4d4-43b8-8b93-ee3135f5f4f6 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:31.772676 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ae594fd6-d4d4-43b8-8b93-ee3135f5f4f6 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:31.786440 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ae594fd6-d4d4-43b8-8b93-ee3135f5f4f6 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:31.787279 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ae594fd6-d4d4-43b8-8b93-ee3135f5f4f6 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:31.791211 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ae594fd6-d4d4-43b8-8b93-ee3135f5f4f6 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:31.791211 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ae594fd6-d4d4-43b8-8b93-ee3135f5f4f6 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:31.791211 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ae594fd6-d4d4-43b8-8b93-ee3135f5f4f6 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:31.848907 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.network.neutron [None req-051114d3-60e5-48bd-9eca-32648aab2ca0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] validate_networks() for [('b16f22e1-6fb1-440b-98eb-845123e960e6', None, None, None, None, None)] {{(pid=121036) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 04 01:07:31.876221 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-ae594fd6-d4d4-43b8-8b93-ee3135f5f4f6 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:31.886466 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-ae594fd6-d4d4-43b8-8b93-ee3135f5f4f6 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5" status: 200 len: 1284 microversion: 2.1 time: 0.125086 Sep 04 01:07:31.887686 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 181/380] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:31 2022] GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 => generated 1284 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:07:31.909483 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-4c78bdf3-6646-490e-896e-7c6fc45fb01a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:31.910099 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-4c78bdf3-6646-490e-896e-7c6fc45fb01a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:31.918621 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4c78bdf3-6646-490e-896e-7c6fc45fb01a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:31.919101 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4c78bdf3-6646-490e-896e-7c6fc45fb01a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:31.922338 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4c78bdf3-6646-490e-896e-7c6fc45fb01a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:31.924441 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4c78bdf3-6646-490e-896e-7c6fc45fb01a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:31.925001 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4c78bdf3-6646-490e-896e-7c6fc45fb01a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:31.925888 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4c78bdf3-6646-490e-896e-7c6fc45fb01a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:32.133255 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-4c78bdf3-6646-490e-896e-7c6fc45fb01a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:32.142709 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-4c78bdf3-6646-490e-896e-7c6fc45fb01a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:32.156953 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-4c78bdf3-6646-490e-896e-7c6fc45fb01a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:32.156953 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-4c78bdf3-6646-490e-896e-7c6fc45fb01a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1721 microversion: 2.20 time: 0.256436 Sep 04 01:07:32.156953 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 182/381] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:07:31 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1721 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:07:32.166067 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-d1827854-d860-40ba-8361-fec7ade95f93 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:32.166817 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-d1827854-d860-40ba-8361-fec7ade95f93 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:32.173484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d1827854-d860-40ba-8361-fec7ade95f93 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:32.174094 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d1827854-d860-40ba-8361-fec7ade95f93 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:32.174639 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d1827854-d860-40ba-8361-fec7ade95f93 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:32.175449 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d1827854-d860-40ba-8361-fec7ade95f93 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:32.175996 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d1827854-d860-40ba-8361-fec7ade95f93 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:32.176511 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d1827854-d860-40ba-8361-fec7ade95f93 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:32.262885 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-051114d3-60e5-48bd-9eca-32648aab2ca0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Flavor limits 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 04 01:07:32.263299 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-051114d3-60e5-48bd-9eca-32648aab2ca0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Image limits 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 04 01:07:32.264076 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-051114d3-60e5-48bd-9eca-32648aab2ca0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Flavor pref 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 04 01:07:32.264481 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-051114d3-60e5-48bd-9eca-32648aab2ca0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Image pref 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 04 01:07:32.264929 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-051114d3-60e5-48bd-9eca-32648aab2ca0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 04 01:07:32.273891 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-051114d3-60e5-48bd-9eca-32648aab2ca0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:32.274368 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-051114d3-60e5-48bd-9eca-32648aab2ca0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:32.274797 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-051114d3-60e5-48bd-9eca-32648aab2ca0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:32.275518 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-051114d3-60e5-48bd-9eca-32648aab2ca0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:32.275959 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-051114d3-60e5-48bd-9eca-32648aab2ca0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:32.276374 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-051114d3-60e5-48bd-9eca-32648aab2ca0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:32.297384 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-051114d3-60e5-48bd-9eca-32648aab2ca0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Getting quotas for project db72b63281384bd883b04d05bcc35677. Resources: {'ram', 'instances', 'cores'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 04 01:07:32.303603 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-051114d3-60e5-48bd-9eca-32648aab2ca0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Getting quotas for user bd13df3a2d0e4182bbd86dab7035c371 and project db72b63281384bd883b04d05bcc35677. Resources: {'ram', 'instances', 'cores'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 04 01:07:32.315114 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-051114d3-60e5-48bd-9eca-32648aab2ca0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Going to run 1 instances... {{(pid=121036) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 04 01:07:32.317434 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-051114d3-60e5-48bd-9eca-32648aab2ca0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a11315c3-ce81-478b-91e4-5353f189dfa7',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121036) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 04 01:07:32.401361 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-d1827854-d860-40ba-8361-fec7ade95f93 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:32.409903 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-d1827854-d860-40ba-8361-fec7ade95f93 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1720 microversion: 2.1 time: 0.251414 Sep 04 01:07:32.410663 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 183/382] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:32 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1720 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:32.429500 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-051114d3-60e5-48bd-9eca-32648aab2ca0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 408 microversion: 2.1 time: 0.703858 Sep 04 01:07:32.430157 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 200/383] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 4 01:07:31 2022] POST /compute/v2.1/servers => generated 408 bytes in 706 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 04 01:07:32.441266 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-43ec0ecc-2881-42ea-96db-767c70e8477b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:32.441990 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-43ec0ecc-2881-42ea-96db-767c70e8477b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:32.504387 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-43ec0ecc-2881-42ea-96db-767c70e8477b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:32.511832 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-43ec0ecc-2881-42ea-96db-767c70e8477b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1246 microversion: 2.1 time: 0.074329 Sep 04 01:07:32.512427 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 184/384] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:32 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1246 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:32.920267 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-68d034de-d67f-4b3e-ab1e-8f7b073745d4 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:32.920985 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-68d034de-d67f-4b3e-ab1e-8f7b073745d4 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:32.927929 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-68d034de-d67f-4b3e-ab1e-8f7b073745d4 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:32.928441 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-68d034de-d67f-4b3e-ab1e-8f7b073745d4 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:32.928840 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-68d034de-d67f-4b3e-ab1e-8f7b073745d4 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:32.929561 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-68d034de-d67f-4b3e-ab1e-8f7b073745d4 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:32.930010 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-68d034de-d67f-4b3e-ab1e-8f7b073745d4 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:32.930429 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-68d034de-d67f-4b3e-ab1e-8f7b073745d4 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:33.056649 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-68d034de-d67f-4b3e-ab1e-8f7b073745d4 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:33.064485 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-68d034de-d67f-4b3e-ab1e-8f7b073745d4 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5" status: 200 len: 1314 microversion: 2.1 time: 0.148525 Sep 04 01:07:33.065100 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 201/385] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:32 2022] GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 => generated 1314 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:33.173055 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-b88362df-7252-4d3f-abf5-8ecfa32289b7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:33.173682 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-b88362df-7252-4d3f-abf5-8ecfa32289b7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:33.181188 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b88362df-7252-4d3f-abf5-8ecfa32289b7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:33.181763 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b88362df-7252-4d3f-abf5-8ecfa32289b7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:33.184422 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b88362df-7252-4d3f-abf5-8ecfa32289b7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:33.184422 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b88362df-7252-4d3f-abf5-8ecfa32289b7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:33.184422 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b88362df-7252-4d3f-abf5-8ecfa32289b7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:33.184422 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b88362df-7252-4d3f-abf5-8ecfa32289b7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:33.322942 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-b88362df-7252-4d3f-abf5-8ecfa32289b7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:33.331204 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-b88362df-7252-4d3f-abf5-8ecfa32289b7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:33.332802 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-b88362df-7252-4d3f-abf5-8ecfa32289b7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:33.334536 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-b88362df-7252-4d3f-abf5-8ecfa32289b7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1721 microversion: 2.20 time: 0.165362 Sep 04 01:07:33.335107 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 185/386] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:07:33 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1721 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:07:33.435968 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-6e25761e-8908-41c0-908b-18c70e7ae5ab tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:33.436579 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-6e25761e-8908-41c0-908b-18c70e7ae5ab tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:33.442588 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6e25761e-8908-41c0-908b-18c70e7ae5ab tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:33.443242 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6e25761e-8908-41c0-908b-18c70e7ae5ab tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:33.443726 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6e25761e-8908-41c0-908b-18c70e7ae5ab tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:33.444412 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6e25761e-8908-41c0-908b-18c70e7ae5ab tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:33.444865 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6e25761e-8908-41c0-908b-18c70e7ae5ab tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:33.445326 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6e25761e-8908-41c0-908b-18c70e7ae5ab tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:33.538572 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-953e385c-6908-4f33-90fe-92d9853fae3a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:33.539183 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-953e385c-6908-4f33-90fe-92d9853fae3a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:33.545100 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-953e385c-6908-4f33-90fe-92d9853fae3a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:33.545644 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-953e385c-6908-4f33-90fe-92d9853fae3a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:33.546079 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-953e385c-6908-4f33-90fe-92d9853fae3a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:33.546797 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-953e385c-6908-4f33-90fe-92d9853fae3a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:33.547246 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-953e385c-6908-4f33-90fe-92d9853fae3a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:33.547665 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-953e385c-6908-4f33-90fe-92d9853fae3a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:33.586981 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-6e25761e-8908-41c0-908b-18c70e7ae5ab tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:33.595141 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-6e25761e-8908-41c0-908b-18c70e7ae5ab tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1720 microversion: 2.1 time: 0.163298 Sep 04 01:07:33.595705 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 202/387] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:33 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1720 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:33.610273 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-953e385c-6908-4f33-90fe-92d9853fae3a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:33.618420 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-953e385c-6908-4f33-90fe-92d9853fae3a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1250 microversion: 2.1 time: 0.084329 Sep 04 01:07:33.619163 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 186/388] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:33 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1250 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:07:34.087816 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-e3807578-a92c-4e4f-98e4-0eb6939b359b tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:34.088419 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-e3807578-a92c-4e4f-98e4-0eb6939b359b tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:34.094849 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e3807578-a92c-4e4f-98e4-0eb6939b359b tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:34.100342 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e3807578-a92c-4e4f-98e4-0eb6939b359b tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:34.100342 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e3807578-a92c-4e4f-98e4-0eb6939b359b tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:34.100342 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e3807578-a92c-4e4f-98e4-0eb6939b359b tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:34.100342 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e3807578-a92c-4e4f-98e4-0eb6939b359b tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:34.100694 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e3807578-a92c-4e4f-98e4-0eb6939b359b tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:34.222486 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e3807578-a92c-4e4f-98e4-0eb6939b359b tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:34.226389 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:34.230901 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-e3807578-a92c-4e4f-98e4-0eb6939b359b tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5" status: 200 len: 1314 microversion: 2.1 time: 0.146972 Sep 04 01:07:34.231462 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 203/389] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:34 2022] GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 => generated 1314 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:34.360021 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-c579af5c-ad5f-416c-b8bd-79f674ac64a0 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:34.360571 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-c579af5c-ad5f-416c-b8bd-79f674ac64a0 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:34.367497 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c579af5c-ad5f-416c-b8bd-79f674ac64a0 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:34.367967 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c579af5c-ad5f-416c-b8bd-79f674ac64a0 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:34.368398 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c579af5c-ad5f-416c-b8bd-79f674ac64a0 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:34.369100 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c579af5c-ad5f-416c-b8bd-79f674ac64a0 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:34.369550 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c579af5c-ad5f-416c-b8bd-79f674ac64a0 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:34.369972 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c579af5c-ad5f-416c-b8bd-79f674ac64a0 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:34.502526 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-c579af5c-ad5f-416c-b8bd-79f674ac64a0 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:34.510687 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-c579af5c-ad5f-416c-b8bd-79f674ac64a0 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:34.512166 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-c579af5c-ad5f-416c-b8bd-79f674ac64a0 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:34.513331 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-c579af5c-ad5f-416c-b8bd-79f674ac64a0 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1721 microversion: 2.20 time: 0.157490 Sep 04 01:07:34.513882 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 187/390] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:07:34 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1721 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:07:34.620705 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-e829a775-57a9-47b0-9c40-fa68ce0a05ba tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:34.621272 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-e829a775-57a9-47b0-9c40-fa68ce0a05ba tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:34.627399 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e829a775-57a9-47b0-9c40-fa68ce0a05ba tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:34.627874 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e829a775-57a9-47b0-9c40-fa68ce0a05ba tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:34.628304 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e829a775-57a9-47b0-9c40-fa68ce0a05ba tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:34.628975 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e829a775-57a9-47b0-9c40-fa68ce0a05ba tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:34.629453 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e829a775-57a9-47b0-9c40-fa68ce0a05ba tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:34.634570 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e829a775-57a9-47b0-9c40-fa68ce0a05ba tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:34.641370 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-865b23ff-d1f2-4258-b345-fdaa4129f252 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:34.641984 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-865b23ff-d1f2-4258-b345-fdaa4129f252 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:34.648050 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-865b23ff-d1f2-4258-b345-fdaa4129f252 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:34.648535 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-865b23ff-d1f2-4258-b345-fdaa4129f252 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:34.648971 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-865b23ff-d1f2-4258-b345-fdaa4129f252 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:34.649702 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-865b23ff-d1f2-4258-b345-fdaa4129f252 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:34.650155 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-865b23ff-d1f2-4258-b345-fdaa4129f252 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:34.650586 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-865b23ff-d1f2-4258-b345-fdaa4129f252 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:34.717215 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-865b23ff-d1f2-4258-b345-fdaa4129f252 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:34.725762 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-865b23ff-d1f2-4258-b345-fdaa4129f252 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1250 microversion: 2.1 time: 0.088161 Sep 04 01:07:34.726332 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 188/391] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:34 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1250 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:07:34.768105 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e829a775-57a9-47b0-9c40-fa68ce0a05ba tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:34.776972 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-e829a775-57a9-47b0-9c40-fa68ce0a05ba tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1720 microversion: 2.1 time: 0.160540 Sep 04 01:07:34.777564 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 204/392] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:34 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1720 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:35.254384 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-3ebadf38-c3e1-4485-bf2d-5e2f02834b4e tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:35.255518 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-3ebadf38-c3e1-4485-bf2d-5e2f02834b4e tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:35.261692 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3ebadf38-c3e1-4485-bf2d-5e2f02834b4e tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:35.262189 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3ebadf38-c3e1-4485-bf2d-5e2f02834b4e tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:35.262609 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3ebadf38-c3e1-4485-bf2d-5e2f02834b4e tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:35.263293 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3ebadf38-c3e1-4485-bf2d-5e2f02834b4e tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:35.263740 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3ebadf38-c3e1-4485-bf2d-5e2f02834b4e tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:35.264162 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3ebadf38-c3e1-4485-bf2d-5e2f02834b4e tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:35.470804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-3ebadf38-c3e1-4485-bf2d-5e2f02834b4e tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:35.478939 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-3ebadf38-c3e1-4485-bf2d-5e2f02834b4e tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5" status: 200 len: 1314 microversion: 2.1 time: 0.228396 Sep 04 01:07:35.479543 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 189/393] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:35 2022] GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 => generated 1314 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:35.539149 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-8516e885-f649-4048-8777-66c7d3c3a742 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:35.539941 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-8516e885-f649-4048-8777-66c7d3c3a742 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:35.546755 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8516e885-f649-4048-8777-66c7d3c3a742 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:35.547237 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8516e885-f649-4048-8777-66c7d3c3a742 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:35.547701 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8516e885-f649-4048-8777-66c7d3c3a742 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:35.548479 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8516e885-f649-4048-8777-66c7d3c3a742 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:35.548929 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8516e885-f649-4048-8777-66c7d3c3a742 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:35.549384 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8516e885-f649-4048-8777-66c7d3c3a742 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:35.684638 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-8516e885-f649-4048-8777-66c7d3c3a742 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:35.693517 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-8516e885-f649-4048-8777-66c7d3c3a742 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:35.695655 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-8516e885-f649-4048-8777-66c7d3c3a742 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:35.696454 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-8516e885-f649-4048-8777-66c7d3c3a742 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1721 microversion: 2.20 time: 0.161372 Sep 04 01:07:35.697011 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 205/394] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:07:35 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1721 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:07:35.749134 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-098ee74e-5666-4422-9f6e-0b70e7124111 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:35.749742 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-098ee74e-5666-4422-9f6e-0b70e7124111 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:35.756208 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-098ee74e-5666-4422-9f6e-0b70e7124111 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:35.756690 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-098ee74e-5666-4422-9f6e-0b70e7124111 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:35.757239 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-098ee74e-5666-4422-9f6e-0b70e7124111 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:35.757848 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-098ee74e-5666-4422-9f6e-0b70e7124111 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:35.758390 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-098ee74e-5666-4422-9f6e-0b70e7124111 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:35.758671 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-098ee74e-5666-4422-9f6e-0b70e7124111 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:35.802709 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-e4134038-e9ec-4e08-9e3a-d400cfb3a604 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:35.803402 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-e4134038-e9ec-4e08-9e3a-d400cfb3a604 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:35.809470 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e4134038-e9ec-4e08-9e3a-d400cfb3a604 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:35.813237 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e4134038-e9ec-4e08-9e3a-d400cfb3a604 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:35.813237 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e4134038-e9ec-4e08-9e3a-d400cfb3a604 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:35.813237 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e4134038-e9ec-4e08-9e3a-d400cfb3a604 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:35.813237 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e4134038-e9ec-4e08-9e3a-d400cfb3a604 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:35.813585 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e4134038-e9ec-4e08-9e3a-d400cfb3a604 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:35.862872 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-098ee74e-5666-4422-9f6e-0b70e7124111 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:35.877216 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-098ee74e-5666-4422-9f6e-0b70e7124111 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1250 microversion: 2.1 time: 0.128601 Sep 04 01:07:35.888875 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 190/395] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:35 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1250 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:07:36.005966 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e4134038-e9ec-4e08-9e3a-d400cfb3a604 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:36.016296 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-e4134038-e9ec-4e08-9e3a-d400cfb3a604 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1720 microversion: 2.1 time: 0.217845 Sep 04 01:07:36.016873 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 206/396] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:35 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1720 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:07:36.125172 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-fe177225-c522-4c8d-839b-7df9d2ea23b5 req-e4572193-beff-46ef-a44b-73c0ab90433c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5", "tag": "b8947ab0-23c1-4eaa-93c4-9c1cc435d43e"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:07:36.135508 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-fe177225-c522-4c8d-839b-7df9d2ea23b5 req-e4572193-beff-46ef-a44b-73c0ab90433c service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:36.136081 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-fe177225-c522-4c8d-839b-7df9d2ea23b5 req-e4572193-beff-46ef-a44b-73c0ab90433c service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:36.136573 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-fe177225-c522-4c8d-839b-7df9d2ea23b5 req-e4572193-beff-46ef-a44b-73c0ab90433c service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:36.148142 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-fe177225-c522-4c8d-839b-7df9d2ea23b5 req-e4572193-beff-46ef-a44b-73c0ab90433c service nova] Creating event network-changed:b8947ab0-23c1-4eaa-93c4-9c1cc435d43e for instance e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 on hv-cinder-maste Sep 04 01:07:36.153552 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-fe177225-c522-4c8d-839b-7df9d2ea23b5 req-e4572193-beff-46ef-a44b-73c0ab90433c service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032383 Sep 04 01:07:36.154144 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 191/397] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:07:36 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:07:36.506738 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-c071f434-d835-441a-adb4-021b677a0813 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:36.507656 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-c071f434-d835-441a-adb4-021b677a0813 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:36.515006 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c071f434-d835-441a-adb4-021b677a0813 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:36.516033 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c071f434-d835-441a-adb4-021b677a0813 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:36.516652 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c071f434-d835-441a-adb4-021b677a0813 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:36.517535 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c071f434-d835-441a-adb4-021b677a0813 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:36.518231 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c071f434-d835-441a-adb4-021b677a0813 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:36.518802 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c071f434-d835-441a-adb4-021b677a0813 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:36.691113 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-c071f434-d835-441a-adb4-021b677a0813 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:36.700128 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-c071f434-d835-441a-adb4-021b677a0813 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5" status: 200 len: 1314 microversion: 2.1 time: 0.201572 Sep 04 01:07:36.700717 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 207/398] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:36 2022] GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 => generated 1314 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:36.724827 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-7fc8ed7c-37d3-4e65-a218-2608e882e8b5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:36.726202 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-7fc8ed7c-37d3-4e65-a218-2608e882e8b5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:36.733190 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7fc8ed7c-37d3-4e65-a218-2608e882e8b5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:36.733683 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7fc8ed7c-37d3-4e65-a218-2608e882e8b5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:36.734128 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7fc8ed7c-37d3-4e65-a218-2608e882e8b5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:36.735097 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7fc8ed7c-37d3-4e65-a218-2608e882e8b5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:36.735556 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7fc8ed7c-37d3-4e65-a218-2608e882e8b5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:36.735983 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7fc8ed7c-37d3-4e65-a218-2608e882e8b5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:36.896717 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-7fc8ed7c-37d3-4e65-a218-2608e882e8b5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:36.900460 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-63614e55-f927-4326-91e9-b1ac0e464f0e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:36.901172 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-63614e55-f927-4326-91e9-b1ac0e464f0e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:36.905262 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-7fc8ed7c-37d3-4e65-a218-2608e882e8b5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:36.907114 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-7fc8ed7c-37d3-4e65-a218-2608e882e8b5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:36.907684 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-63614e55-f927-4326-91e9-b1ac0e464f0e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:36.908239 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-63614e55-f927-4326-91e9-b1ac0e464f0e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:36.908734 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-7fc8ed7c-37d3-4e65-a218-2608e882e8b5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1721 microversion: 2.20 time: 0.189366 Sep 04 01:07:36.908846 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-63614e55-f927-4326-91e9-b1ac0e464f0e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:36.909532 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-63614e55-f927-4326-91e9-b1ac0e464f0e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:36.910050 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-63614e55-f927-4326-91e9-b1ac0e464f0e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:36.910830 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-63614e55-f927-4326-91e9-b1ac0e464f0e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:36.921698 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 192/399] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:07:36 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1721 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:07:36.981621 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-63614e55-f927-4326-91e9-b1ac0e464f0e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:36.992592 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-63614e55-f927-4326-91e9-b1ac0e464f0e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1298 microversion: 2.1 time: 0.096811 Sep 04 01:07:36.993208 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 208/400] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:36 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1298 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:37.040019 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-eb681bf1-fca7-40c2-b243-64ab1630eecf tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:37.040711 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-eb681bf1-fca7-40c2-b243-64ab1630eecf tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:37.046610 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-eb681bf1-fca7-40c2-b243-64ab1630eecf tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:37.047215 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-eb681bf1-fca7-40c2-b243-64ab1630eecf tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:37.047650 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-eb681bf1-fca7-40c2-b243-64ab1630eecf tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:37.048335 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-eb681bf1-fca7-40c2-b243-64ab1630eecf tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:37.048784 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-eb681bf1-fca7-40c2-b243-64ab1630eecf tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:37.049231 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-eb681bf1-fca7-40c2-b243-64ab1630eecf tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:37.286476 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-eb681bf1-fca7-40c2-b243-64ab1630eecf tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:37.297525 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-eb681bf1-fca7-40c2-b243-64ab1630eecf tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1720 microversion: 2.1 time: 0.261614 Sep 04 01:07:37.298434 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 193/401] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:37 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1720 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:37.723524 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-c581539c-c484-4315-8327-1f76282f5690 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:37.724308 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-c581539c-c484-4315-8327-1f76282f5690 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:37.730746 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c581539c-c484-4315-8327-1f76282f5690 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:37.731216 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c581539c-c484-4315-8327-1f76282f5690 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:37.731648 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c581539c-c484-4315-8327-1f76282f5690 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:37.732333 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c581539c-c484-4315-8327-1f76282f5690 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:37.732783 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c581539c-c484-4315-8327-1f76282f5690 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:37.733242 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c581539c-c484-4315-8327-1f76282f5690 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:38.021963 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-c581539c-c484-4315-8327-1f76282f5690 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:38.021963 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-c581539c-c484-4315-8327-1f76282f5690 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5" status: 200 len: 1314 microversion: 2.1 time: 0.170739 Sep 04 01:07:38.021963 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 209/402] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:37 2022] GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 => generated 1314 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:38.025878 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-fd620f86-351f-4e7d-8bab-00be1bcd841d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:38.025878 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-fd620f86-351f-4e7d-8bab-00be1bcd841d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:38.025878 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fd620f86-351f-4e7d-8bab-00be1bcd841d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:38.025878 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fd620f86-351f-4e7d-8bab-00be1bcd841d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:38.027548 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fd620f86-351f-4e7d-8bab-00be1bcd841d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:38.027548 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fd620f86-351f-4e7d-8bab-00be1bcd841d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:38.027548 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fd620f86-351f-4e7d-8bab-00be1bcd841d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:38.027548 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fd620f86-351f-4e7d-8bab-00be1bcd841d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:38.027970 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-d001310b-387e-48bc-8cd4-9028a9d616f0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:38.027970 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-d001310b-387e-48bc-8cd4-9028a9d616f0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:38.033164 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d001310b-387e-48bc-8cd4-9028a9d616f0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:38.033663 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d001310b-387e-48bc-8cd4-9028a9d616f0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:38.034098 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d001310b-387e-48bc-8cd4-9028a9d616f0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:38.034804 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d001310b-387e-48bc-8cd4-9028a9d616f0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:38.035290 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d001310b-387e-48bc-8cd4-9028a9d616f0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:38.035717 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d001310b-387e-48bc-8cd4-9028a9d616f0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:38.107111 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-d001310b-387e-48bc-8cd4-9028a9d616f0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:38.109989 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-fd620f86-351f-4e7d-8bab-00be1bcd841d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:38.116614 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-d001310b-387e-48bc-8cd4-9028a9d616f0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1298 microversion: 2.1 time: 0.102674 Sep 04 01:07:38.117221 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 210/403] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:38 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1298 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:38.119321 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-fd620f86-351f-4e7d-8bab-00be1bcd841d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:38.120856 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-fd620f86-351f-4e7d-8bab-00be1bcd841d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:38.122246 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-fd620f86-351f-4e7d-8bab-00be1bcd841d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1721 microversion: 2.20 time: 0.190525 Sep 04 01:07:38.123014 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 194/404] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:07:37 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1721 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:07:38.158450 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-fd827f61-0430-4232-81e8-222ddb5cd2da tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Action: 'create', calling method: .version_select at 0x7fc10c71b8b0>, body: {"keypair": {"name": "tempest-keypair-1542700883", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFrewL5xUAcGPzk7Gbqasr9lwQeoY42o/ByQ2CMTCLYGw15oddJfeX4D46TfrONGVoQYxq4a17uA/Aq6Hrd2KldCItQ/Cvv7nBllGtC3qM1pzXnROWcQR5ZdTC/IYViWRw=="}} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:07:38.174422 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-fd827f61-0430-4232-81e8-222ddb5cd2da tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Getting quotas for project 287d111f08ac4b05bd5adec9c40d0620. Resources: {'key_pairs'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 04 01:07:38.178959 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-fd827f61-0430-4232-81e8-222ddb5cd2da tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Getting quotas for user f5e4723258c64ce5885154dded2d3d43 and project 287d111f08ac4b05bd5adec9c40d0620. Resources: {'key_pairs'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 04 01:07:38.315282 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-fd827f61-0430-4232-81e8-222ddb5cd2da tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.160592 Sep 04 01:07:38.318112 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 211/405] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Sep 4 01:07:38 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:07:38.326445 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-35c3dc23-7615-409d-b908-0ceb8c3e2419 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:38.327582 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-35c3dc23-7615-409d-b908-0ceb8c3e2419 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:38.331062 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell0:SELECT=2 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:38.339972 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-35c3dc23-7615-409d-b908-0ceb8c3e2419 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:38.340881 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-35c3dc23-7615-409d-b908-0ceb8c3e2419 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:38.341688 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-35c3dc23-7615-409d-b908-0ceb8c3e2419 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:38.342933 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-35c3dc23-7615-409d-b908-0ceb8c3e2419 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:38.343760 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-35c3dc23-7615-409d-b908-0ceb8c3e2419 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:38.344524 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-35c3dc23-7615-409d-b908-0ceb8c3e2419 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:38.488875 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-35c3dc23-7615-409d-b908-0ceb8c3e2419 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:38.498316 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-35c3dc23-7615-409d-b908-0ceb8c3e2419 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1720 microversion: 2.1 time: 0.178828 Sep 04 01:07:38.499311 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 195/406] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:38 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1720 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:39.037550 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-1a982a56-758c-4a0f-8466-0a0152a0ff28 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:39.038193 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-1a982a56-758c-4a0f-8466-0a0152a0ff28 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:39.044424 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1a982a56-758c-4a0f-8466-0a0152a0ff28 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:39.044911 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1a982a56-758c-4a0f-8466-0a0152a0ff28 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:39.045374 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1a982a56-758c-4a0f-8466-0a0152a0ff28 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:39.046067 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1a982a56-758c-4a0f-8466-0a0152a0ff28 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:39.046518 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1a982a56-758c-4a0f-8466-0a0152a0ff28 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:39.046948 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1a982a56-758c-4a0f-8466-0a0152a0ff28 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:39.139670 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-7a31f4a6-988f-48ea-8845-b5ebe035909a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:39.140277 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-7a31f4a6-988f-48ea-8845-b5ebe035909a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:39.147907 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7a31f4a6-988f-48ea-8845-b5ebe035909a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:39.148386 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7a31f4a6-988f-48ea-8845-b5ebe035909a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:39.148816 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7a31f4a6-988f-48ea-8845-b5ebe035909a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:39.149716 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7a31f4a6-988f-48ea-8845-b5ebe035909a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:39.150636 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7a31f4a6-988f-48ea-8845-b5ebe035909a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:39.154733 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7a31f4a6-988f-48ea-8845-b5ebe035909a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:39.212807 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-7a31f4a6-988f-48ea-8845-b5ebe035909a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:39.212984 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-1a982a56-758c-4a0f-8466-0a0152a0ff28 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:39.220714 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-7a31f4a6-988f-48ea-8845-b5ebe035909a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1306 microversion: 2.1 time: 0.084785 Sep 04 01:07:39.220886 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-1a982a56-758c-4a0f-8466-0a0152a0ff28 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5" status: 200 len: 1314 microversion: 2.1 time: 0.188327 Sep 04 01:07:39.221310 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 196/407] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:39 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1306 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:39.237061 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 212/408] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:39 2022] GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 => generated 1314 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:39.237061 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-4a8d3391-53bf-4b53-9d81-b43b866513ef tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:39.237061 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-4a8d3391-53bf-4b53-9d81-b43b866513ef tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:39.237061 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4a8d3391-53bf-4b53-9d81-b43b866513ef tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:39.237460 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4a8d3391-53bf-4b53-9d81-b43b866513ef tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:39.237460 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4a8d3391-53bf-4b53-9d81-b43b866513ef tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:39.237460 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4a8d3391-53bf-4b53-9d81-b43b866513ef tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:39.237460 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4a8d3391-53bf-4b53-9d81-b43b866513ef tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:39.237743 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4a8d3391-53bf-4b53-9d81-b43b866513ef tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:39.371528 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-4a8d3391-53bf-4b53-9d81-b43b866513ef tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:39.380745 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-4a8d3391-53bf-4b53-9d81-b43b866513ef tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:39.381367 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-4a8d3391-53bf-4b53-9d81-b43b866513ef tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:39.382530 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-4a8d3391-53bf-4b53-9d81-b43b866513ef tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1721 microversion: 2.20 time: 0.160223 Sep 04 01:07:39.383073 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 213/409] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:07:39 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1721 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:07:39.523059 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-06b4a5fe-531f-4414-aa90-86597ca8a638 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:39.523516 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-06b4a5fe-531f-4414-aa90-86597ca8a638 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:39.530769 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-06b4a5fe-531f-4414-aa90-86597ca8a638 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:39.532668 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-06b4a5fe-531f-4414-aa90-86597ca8a638 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:39.533370 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-06b4a5fe-531f-4414-aa90-86597ca8a638 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:39.534444 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-06b4a5fe-531f-4414-aa90-86597ca8a638 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:39.535160 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-06b4a5fe-531f-4414-aa90-86597ca8a638 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:39.536641 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-06b4a5fe-531f-4414-aa90-86597ca8a638 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:39.757656 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-06b4a5fe-531f-4414-aa90-86597ca8a638 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:39.770449 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-06b4a5fe-531f-4414-aa90-86597ca8a638 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1720 microversion: 2.1 time: 0.252066 Sep 04 01:07:39.772859 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 197/410] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:39 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1720 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:40.247222 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-81e11704-6249-4846-a08d-f8f168591538 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:40.248820 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-81e11704-6249-4846-a08d-f8f168591538 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:40.254095 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-83bae6cd-d7da-437a-96d3-a346f883e13b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:40.254095 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-83bae6cd-d7da-437a-96d3-a346f883e13b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:40.259867 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-81e11704-6249-4846-a08d-f8f168591538 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:40.260693 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-81e11704-6249-4846-a08d-f8f168591538 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:40.261407 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-81e11704-6249-4846-a08d-f8f168591538 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:40.262099 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-83bae6cd-d7da-437a-96d3-a346f883e13b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:40.262560 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-81e11704-6249-4846-a08d-f8f168591538 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:40.263027 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-83bae6cd-d7da-437a-96d3-a346f883e13b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:40.263681 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-81e11704-6249-4846-a08d-f8f168591538 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:40.264309 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-81e11704-6249-4846-a08d-f8f168591538 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:40.264680 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-83bae6cd-d7da-437a-96d3-a346f883e13b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:40.265782 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-83bae6cd-d7da-437a-96d3-a346f883e13b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:40.266371 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-83bae6cd-d7da-437a-96d3-a346f883e13b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:40.266932 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-83bae6cd-d7da-437a-96d3-a346f883e13b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:40.296903 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=100 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:40.524010 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-81e11704-6249-4846-a08d-f8f168591538 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:40.534797 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-81e11704-6249-4846-a08d-f8f168591538 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5" status: 200 len: 1314 microversion: 2.1 time: 0.291684 Sep 04 01:07:40.545349 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 198/411] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:40 2022] GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 => generated 1314 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:40.545349 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-c65d4c68-7d97-48f5-8b95-9be37a5147d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:40.545687 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-c65d4c68-7d97-48f5-8b95-9be37a5147d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:40.553743 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c65d4c68-7d97-48f5-8b95-9be37a5147d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:40.554473 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c65d4c68-7d97-48f5-8b95-9be37a5147d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:40.555058 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c65d4c68-7d97-48f5-8b95-9be37a5147d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:40.556053 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c65d4c68-7d97-48f5-8b95-9be37a5147d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:40.556636 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c65d4c68-7d97-48f5-8b95-9be37a5147d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:40.557258 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c65d4c68-7d97-48f5-8b95-9be37a5147d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:40.691389 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-83bae6cd-d7da-437a-96d3-a346f883e13b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:40.710707 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-83bae6cd-d7da-437a-96d3-a346f883e13b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1371 microversion: 2.1 time: 0.468670 Sep 04 01:07:40.712822 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 214/412] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:40 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1371 bytes in 472 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:40.799773 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-d82ff6bc-81e6-4379-8206-6029e6ed86a3 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:40.802631 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-d82ff6bc-81e6-4379-8206-6029e6ed86a3 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:40.811771 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d82ff6bc-81e6-4379-8206-6029e6ed86a3 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:40.812438 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d82ff6bc-81e6-4379-8206-6029e6ed86a3 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:40.816490 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-c65d4c68-7d97-48f5-8b95-9be37a5147d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:40.827005 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d82ff6bc-81e6-4379-8206-6029e6ed86a3 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:40.828128 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d82ff6bc-81e6-4379-8206-6029e6ed86a3 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:40.828812 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d82ff6bc-81e6-4379-8206-6029e6ed86a3 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:40.830083 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-c65d4c68-7d97-48f5-8b95-9be37a5147d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:40.830597 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d82ff6bc-81e6-4379-8206-6029e6ed86a3 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:40.834405 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-c65d4c68-7d97-48f5-8b95-9be37a5147d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:40.838544 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-c65d4c68-7d97-48f5-8b95-9be37a5147d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1721 microversion: 2.20 time: 0.302010 Sep 04 01:07:40.839516 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 199/413] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:07:40 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1721 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:07:41.068991 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-d82ff6bc-81e6-4379-8206-6029e6ed86a3 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:41.071077 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-ddfe3e84-476a-4635-8fa5-0dc3855a3f9a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-2121083598", "imageRef": "a11315c3-ce81-478b-91e4-5353f189dfa7", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "d7f39628-666e-4480-8bee-49de183d5e09"}], "security_groups": [{"name": "tempest-securitygroup--1809104088"}], "key_name": "tempest-keypair-1542700883", "user_data": "}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:07:41.078687 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-d82ff6bc-81e6-4379-8206-6029e6ed86a3 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1720 microversion: 2.1 time: 0.285229 Sep 04 01:07:41.079707 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 215/414] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:40 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1720 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:41.185362 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.network.neutron [None req-ddfe3e84-476a-4635-8fa5-0dc3855a3f9a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] validate_networks() for [('d7f39628-666e-4480-8bee-49de183d5e09', None, None, None, None, None)] {{(pid=121037) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 04 01:07:41.564843 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-d250b8f2-0805-4b20-b9ac-b047e576e2d5 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:41.565628 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-d250b8f2-0805-4b20-b9ac-b047e576e2d5 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:41.572627 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d250b8f2-0805-4b20-b9ac-b047e576e2d5 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:41.573143 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d250b8f2-0805-4b20-b9ac-b047e576e2d5 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:41.573588 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d250b8f2-0805-4b20-b9ac-b047e576e2d5 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:41.574349 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d250b8f2-0805-4b20-b9ac-b047e576e2d5 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:41.574804 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d250b8f2-0805-4b20-b9ac-b047e576e2d5 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:41.575528 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d250b8f2-0805-4b20-b9ac-b047e576e2d5 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:41.714047 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-d250b8f2-0805-4b20-b9ac-b047e576e2d5 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:41.722067 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-d250b8f2-0805-4b20-b9ac-b047e576e2d5 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5" status: 200 len: 1314 microversion: 2.1 time: 0.161328 Sep 04 01:07:41.724285 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 216/415] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:41 2022] GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 => generated 1314 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:07:41.730324 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-44f4281c-6406-408c-a782-706be53a6d55 req-bd3af84b-62a7-4d5f-b32f-592d01b850cc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2570ee6c-f335-41bc-8e67-f89a62b28134", "tag": "62f5079a-1d7d-4c5c-bdcd-0af337b6235b"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:07:41.740221 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-44f4281c-6406-408c-a782-706be53a6d55 req-bd3af84b-62a7-4d5f-b32f-592d01b850cc service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:41.740771 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-44f4281c-6406-408c-a782-706be53a6d55 req-bd3af84b-62a7-4d5f-b32f-592d01b850cc service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:41.741303 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-44f4281c-6406-408c-a782-706be53a6d55 req-bd3af84b-62a7-4d5f-b32f-592d01b850cc service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:41.759155 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-ddfe3e84-476a-4635-8fa5-0dc3855a3f9a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Flavor limits 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 04 01:07:41.759645 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-44f4281c-6406-408c-a782-706be53a6d55 req-bd3af84b-62a7-4d5f-b32f-592d01b850cc service nova] Creating event network-changed:62f5079a-1d7d-4c5c-bdcd-0af337b6235b for instance 2570ee6c-f335-41bc-8e67-f89a62b28134 on hv-cinder-maste Sep 04 01:07:41.759753 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-ddfe3e84-476a-4635-8fa5-0dc3855a3f9a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Image limits 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 04 01:07:41.760049 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-ddfe3e84-476a-4635-8fa5-0dc3855a3f9a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Flavor pref 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 04 01:07:41.760447 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-ddfe3e84-476a-4635-8fa5-0dc3855a3f9a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Image pref 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 04 01:07:41.760942 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-ddfe3e84-476a-4635-8fa5-0dc3855a3f9a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 04 01:07:41.764426 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-44f4281c-6406-408c-a782-706be53a6d55 req-bd3af84b-62a7-4d5f-b32f-592d01b850cc service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040720 Sep 04 01:07:41.765049 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 217/416] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:07:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:07:41.770266 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-f9bf936d-185f-40a5-b79f-d273f0e7313f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:41.771173 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-f9bf936d-185f-40a5-b79f-d273f0e7313f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:41.771860 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ddfe3e84-476a-4635-8fa5-0dc3855a3f9a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:41.772336 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ddfe3e84-476a-4635-8fa5-0dc3855a3f9a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:41.772769 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ddfe3e84-476a-4635-8fa5-0dc3855a3f9a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:41.773524 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ddfe3e84-476a-4635-8fa5-0dc3855a3f9a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:41.777339 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ddfe3e84-476a-4635-8fa5-0dc3855a3f9a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:41.777948 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ddfe3e84-476a-4635-8fa5-0dc3855a3f9a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:41.778228 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f9bf936d-185f-40a5-b79f-d273f0e7313f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:41.778806 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f9bf936d-185f-40a5-b79f-d273f0e7313f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:41.779314 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f9bf936d-185f-40a5-b79f-d273f0e7313f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:41.780121 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f9bf936d-185f-40a5-b79f-d273f0e7313f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:41.780692 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f9bf936d-185f-40a5-b79f-d273f0e7313f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:41.781203 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f9bf936d-185f-40a5-b79f-d273f0e7313f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:41.795532 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.quota [None req-ddfe3e84-476a-4635-8fa5-0dc3855a3f9a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Getting quotas for project 287d111f08ac4b05bd5adec9c40d0620. Resources: {'ram', 'instances', 'cores'} {{(pid=121037) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 04 01:07:41.799745 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.quota [None req-ddfe3e84-476a-4635-8fa5-0dc3855a3f9a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Getting quotas for user f5e4723258c64ce5885154dded2d3d43 and project 287d111f08ac4b05bd5adec9c40d0620. Resources: {'ram', 'instances', 'cores'} {{(pid=121037) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 04 01:07:41.811922 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-ddfe3e84-476a-4635-8fa5-0dc3855a3f9a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Going to run 1 instances... {{(pid=121037) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 04 01:07:41.814654 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-ddfe3e84-476a-4635-8fa5-0dc3855a3f9a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a11315c3-ce81-478b-91e4-5353f189dfa7',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121037) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 04 01:07:41.914254 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-f9bf936d-185f-40a5-b79f-d273f0e7313f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:41.922028 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-f9bf936d-185f-40a5-b79f-d273f0e7313f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1371 microversion: 2.1 time: 0.155721 Sep 04 01:07:41.923764 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 218/417] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:41 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1371 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:41.927429 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-3c980188-dce9-40bb-9595-24f6a81e342b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:41.928021 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-3c980188-dce9-40bb-9595-24f6a81e342b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:41.933638 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3c980188-dce9-40bb-9595-24f6a81e342b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:41.934115 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3c980188-dce9-40bb-9595-24f6a81e342b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:41.934546 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3c980188-dce9-40bb-9595-24f6a81e342b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:41.935228 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3c980188-dce9-40bb-9595-24f6a81e342b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:41.935677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3c980188-dce9-40bb-9595-24f6a81e342b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:41.936100 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3c980188-dce9-40bb-9595-24f6a81e342b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:41.963932 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-ddfe3e84-476a-4635-8fa5-0dc3855a3f9a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.896438 Sep 04 01:07:41.964147 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 200/418] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 4 01:07:41 2022] POST /compute/v2.1/servers => generated 402 bytes in 899 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 04 01:07:41.976037 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-f28c20e9-9a22-4937-8267-98f430805915 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:41.976660 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-f28c20e9-9a22-4937-8267-98f430805915 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:42.044504 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-f28c20e9-9a22-4937-8267-98f430805915 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:42.053493 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-f28c20e9-9a22-4937-8267-98f430805915 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1242 microversion: 2.1 time: 0.082103 Sep 04 01:07:42.069225 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 201/419] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:41 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1242 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:07:42.093573 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-3c980188-dce9-40bb-9595-24f6a81e342b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:42.128204 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-3c980188-dce9-40bb-9595-24f6a81e342b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:42.128204 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-3c980188-dce9-40bb-9595-24f6a81e342b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:42.131136 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-099c8381-6ce9-4efe-9a75-168c6cbbd23c tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:42.132124 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-099c8381-6ce9-4efe-9a75-168c6cbbd23c tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:42.133832 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-3c980188-dce9-40bb-9595-24f6a81e342b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1721 microversion: 2.20 time: 0.210260 Sep 04 01:07:42.136153 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 219/420] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:07:41 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1721 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:07:42.163514 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-099c8381-6ce9-4efe-9a75-168c6cbbd23c tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:42.165281 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-099c8381-6ce9-4efe-9a75-168c6cbbd23c tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:42.165656 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-099c8381-6ce9-4efe-9a75-168c6cbbd23c tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:42.166463 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-099c8381-6ce9-4efe-9a75-168c6cbbd23c tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:42.178756 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-099c8381-6ce9-4efe-9a75-168c6cbbd23c tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:42.178756 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-099c8381-6ce9-4efe-9a75-168c6cbbd23c tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:42.394882 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-7d0c5ede-f050-4704-9ef3-c34744b678d6 req-6da9bf9a-bc8e-48c0-93b1-0da43fd3c75a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5", "name": "network-vif-plugged", "status": "completed", "tag": "b8947ab0-23c1-4eaa-93c4-9c1cc435d43e"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:07:42.401179 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell0:SELECT=2 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:42.407666 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-7d0c5ede-f050-4704-9ef3-c34744b678d6 req-6da9bf9a-bc8e-48c0-93b1-0da43fd3c75a service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:42.408344 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-7d0c5ede-f050-4704-9ef3-c34744b678d6 req-6da9bf9a-bc8e-48c0-93b1-0da43fd3c75a service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:42.408955 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-7d0c5ede-f050-4704-9ef3-c34744b678d6 req-6da9bf9a-bc8e-48c0-93b1-0da43fd3c75a service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:42.422725 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-7d0c5ede-f050-4704-9ef3-c34744b678d6 req-6da9bf9a-bc8e-48c0-93b1-0da43fd3c75a service nova] Creating event network-vif-plugged:b8947ab0-23c1-4eaa-93c4-9c1cc435d43e for instance e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 on hv-cinder-maste Sep 04 01:07:42.434680 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-7d0c5ede-f050-4704-9ef3-c34744b678d6 req-6da9bf9a-bc8e-48c0-93b1-0da43fd3c75a service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043967 Sep 04 01:07:42.435623 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 220/421] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:07:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:07:42.693293 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-099c8381-6ce9-4efe-9a75-168c6cbbd23c tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:42.705191 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-099c8381-6ce9-4efe-9a75-168c6cbbd23c tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1720 microversion: 2.1 time: 0.595711 Sep 04 01:07:42.705903 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 202/422] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:42 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1720 bytes in 597 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:42.745949 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-5165cc0f-2ab9-4ba8-9c4d-8982deb44d51 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:42.747262 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-5165cc0f-2ab9-4ba8-9c4d-8982deb44d51 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:42.753999 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5165cc0f-2ab9-4ba8-9c4d-8982deb44d51 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:42.757604 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5165cc0f-2ab9-4ba8-9c4d-8982deb44d51 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:42.757604 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5165cc0f-2ab9-4ba8-9c4d-8982deb44d51 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:42.757604 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5165cc0f-2ab9-4ba8-9c4d-8982deb44d51 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:42.757604 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5165cc0f-2ab9-4ba8-9c4d-8982deb44d51 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:42.757956 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5165cc0f-2ab9-4ba8-9c4d-8982deb44d51 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:42.909599 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-5165cc0f-2ab9-4ba8-9c4d-8982deb44d51 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:42.917880 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-5165cc0f-2ab9-4ba8-9c4d-8982deb44d51 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5" status: 200 len: 1314 microversion: 2.1 time: 0.176111 Sep 04 01:07:42.918659 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 221/423] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:42 2022] GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 => generated 1314 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:42.946176 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-76c5dfbf-14ba-4f38-9cc9-abae557c8b15 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:42.946790 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-76c5dfbf-14ba-4f38-9cc9-abae557c8b15 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:42.952717 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-76c5dfbf-14ba-4f38-9cc9-abae557c8b15 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:42.953242 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-76c5dfbf-14ba-4f38-9cc9-abae557c8b15 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:42.953685 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-76c5dfbf-14ba-4f38-9cc9-abae557c8b15 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:42.954374 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-76c5dfbf-14ba-4f38-9cc9-abae557c8b15 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:42.954828 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-76c5dfbf-14ba-4f38-9cc9-abae557c8b15 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:42.955254 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-76c5dfbf-14ba-4f38-9cc9-abae557c8b15 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:43.076240 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-ba31ae21-5b1c-461c-8a23-0b1cb4048a02 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:43.077033 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-ba31ae21-5b1c-461c-8a23-0b1cb4048a02 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:43.083662 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ba31ae21-5b1c-461c-8a23-0b1cb4048a02 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:43.084213 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ba31ae21-5b1c-461c-8a23-0b1cb4048a02 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:43.086617 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ba31ae21-5b1c-461c-8a23-0b1cb4048a02 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:43.086617 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ba31ae21-5b1c-461c-8a23-0b1cb4048a02 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:43.086617 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ba31ae21-5b1c-461c-8a23-0b1cb4048a02 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:43.086617 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ba31ae21-5b1c-461c-8a23-0b1cb4048a02 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:43.154243 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ba31ae21-5b1c-461c-8a23-0b1cb4048a02 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:43.158180 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=100 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:43.159501 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-76c5dfbf-14ba-4f38-9cc9-abae557c8b15 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:43.162910 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-ba31ae21-5b1c-461c-8a23-0b1cb4048a02 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1246 microversion: 2.1 time: 0.090821 Sep 04 01:07:43.163757 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 222/424] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:43 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1246 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:43.168546 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-aaecf7fc-51cf-4e7c-8251-7fb06a95b80b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:43.169197 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-aaecf7fc-51cf-4e7c-8251-7fb06a95b80b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:43.181415 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-aaecf7fc-51cf-4e7c-8251-7fb06a95b80b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:43.182034 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-aaecf7fc-51cf-4e7c-8251-7fb06a95b80b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:43.182488 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-76c5dfbf-14ba-4f38-9cc9-abae557c8b15 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1371 microversion: 2.1 time: 0.240709 Sep 04 01:07:43.182750 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-aaecf7fc-51cf-4e7c-8251-7fb06a95b80b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:43.183294 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 203/425] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:42 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1371 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:43.183938 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-aaecf7fc-51cf-4e7c-8251-7fb06a95b80b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:43.184514 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-aaecf7fc-51cf-4e7c-8251-7fb06a95b80b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:43.185333 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-aaecf7fc-51cf-4e7c-8251-7fb06a95b80b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:43.334314 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-aaecf7fc-51cf-4e7c-8251-7fb06a95b80b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:43.345441 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-aaecf7fc-51cf-4e7c-8251-7fb06a95b80b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:43.347074 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-aaecf7fc-51cf-4e7c-8251-7fb06a95b80b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:43.348617 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-aaecf7fc-51cf-4e7c-8251-7fb06a95b80b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1742 microversion: 2.20 time: 0.183470 Sep 04 01:07:43.349301 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 223/426] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:07:43 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1742 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:07:43.730098 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-4b768005-f9c2-40be-a20c-6bb92f6a8825 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:43.730703 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-4b768005-f9c2-40be-a20c-6bb92f6a8825 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:43.737476 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4b768005-f9c2-40be-a20c-6bb92f6a8825 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:43.737959 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4b768005-f9c2-40be-a20c-6bb92f6a8825 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:43.738393 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4b768005-f9c2-40be-a20c-6bb92f6a8825 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:43.739744 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4b768005-f9c2-40be-a20c-6bb92f6a8825 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:43.740183 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4b768005-f9c2-40be-a20c-6bb92f6a8825 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:43.740608 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4b768005-f9c2-40be-a20c-6bb92f6a8825 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:43.919948 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-4b768005-f9c2-40be-a20c-6bb92f6a8825 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:43.928688 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-4b768005-f9c2-40be-a20c-6bb92f6a8825 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1697 microversion: 2.1 time: 0.202343 Sep 04 01:07:43.929328 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 204/427] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:43 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1697 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:43.940452 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-cfbbf090-2fa1-4289-9cc8-b81350336851 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:43.941062 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-cfbbf090-2fa1-4289-9cc8-b81350336851 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:43.947605 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cfbbf090-2fa1-4289-9cc8-b81350336851 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:43.948126 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cfbbf090-2fa1-4289-9cc8-b81350336851 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:43.948533 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cfbbf090-2fa1-4289-9cc8-b81350336851 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:43.949296 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cfbbf090-2fa1-4289-9cc8-b81350336851 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:43.949749 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cfbbf090-2fa1-4289-9cc8-b81350336851 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:43.950172 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cfbbf090-2fa1-4289-9cc8-b81350336851 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:43.951887 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-fd68b874-7556-46e2-8141-24103ec45033 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:07:43.960791 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fd68b874-7556-46e2-8141-24103ec45033 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:43.961322 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fd68b874-7556-46e2-8141-24103ec45033 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:43.961762 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fd68b874-7556-46e2-8141-24103ec45033 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:43.988695 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-fd68b874-7556-46e2-8141-24103ec45033 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Going to try to start instance {{(pid=121037) start /opt/stack/nova/nova/compute/api.py:2785}} Sep 04 01:07:44.080920 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-cfbbf090-2fa1-4289-9cc8-b81350336851 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:44.089438 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-cfbbf090-2fa1-4289-9cc8-b81350336851 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5" status: 200 len: 1314 microversion: 2.1 time: 0.152812 Sep 04 01:07:44.090006 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 224/428] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:43 2022] GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 => generated 1314 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:07:44.187518 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-84363e30-1136-4fb0-9681-f2b93d2d08b4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:44.189813 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-84363e30-1136-4fb0-9681-f2b93d2d08b4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:44.189988 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=4 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:44.195180 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-84363e30-1136-4fb0-9681-f2b93d2d08b4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:44.195663 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-84363e30-1136-4fb0-9681-f2b93d2d08b4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:44.196100 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-84363e30-1136-4fb0-9681-f2b93d2d08b4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:44.196796 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-84363e30-1136-4fb0-9681-f2b93d2d08b4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:44.197269 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-84363e30-1136-4fb0-9681-f2b93d2d08b4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:44.197692 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-84363e30-1136-4fb0-9681-f2b93d2d08b4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:44.213854 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-fd68b874-7556-46e2-8141-24103ec45033 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "POST /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2/action" status: 202 len: 0 microversion: 2.1 time: 0.265363 Sep 04 01:07:44.214862 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 205/429] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Sep 4 01:07:43 2022] POST /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2/action => generated 0 bytes in 267 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 04 01:07:44.220362 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-3fa591eb-02fd-47ed-b3e1-fd2fddebbe82 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:44.221212 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-3fa591eb-02fd-47ed-b3e1-fd2fddebbe82 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:44.227917 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3fa591eb-02fd-47ed-b3e1-fd2fddebbe82 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:44.228693 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3fa591eb-02fd-47ed-b3e1-fd2fddebbe82 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:44.229222 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3fa591eb-02fd-47ed-b3e1-fd2fddebbe82 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:44.230049 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3fa591eb-02fd-47ed-b3e1-fd2fddebbe82 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:44.230559 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3fa591eb-02fd-47ed-b3e1-fd2fddebbe82 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:44.231046 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3fa591eb-02fd-47ed-b3e1-fd2fddebbe82 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:44.268291 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-84363e30-1136-4fb0-9681-f2b93d2d08b4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:44.278938 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-84363e30-1136-4fb0-9681-f2b93d2d08b4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1238 microversion: 2.1 time: 0.095301 Sep 04 01:07:44.287766 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 225/430] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:44 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1238 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:07:44.287766 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-e14fc244-8761-4b7c-a135-92bb61feb76a tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:44.287766 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-e14fc244-8761-4b7c-a135-92bb61feb76a tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:44.291949 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e14fc244-8761-4b7c-a135-92bb61feb76a tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:44.292431 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e14fc244-8761-4b7c-a135-92bb61feb76a tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:44.293159 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e14fc244-8761-4b7c-a135-92bb61feb76a tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:44.294192 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e14fc244-8761-4b7c-a135-92bb61feb76a tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:44.294775 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e14fc244-8761-4b7c-a135-92bb61feb76a tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:44.295359 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e14fc244-8761-4b7c-a135-92bb61feb76a tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:44.493383 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-3fa591eb-02fd-47ed-b3e1-fd2fddebbe82 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:44.501317 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e14fc244-8761-4b7c-a135-92bb61feb76a tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:44.503856 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-3fa591eb-02fd-47ed-b3e1-fd2fddebbe82 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1371 microversion: 2.1 time: 0.287929 Sep 04 01:07:44.511738 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 206/431] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:44 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1371 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:44.511994 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-e14fc244-8761-4b7c-a135-92bb61feb76a tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1706 microversion: 2.1 time: 0.231411 Sep 04 01:07:44.512863 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 226/432] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:44 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1706 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:44.518036 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-fdb37004-2d3f-436f-b6a8-4586ac2dfe4c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:44.518167 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-fdb37004-2d3f-436f-b6a8-4586ac2dfe4c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:44.533949 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fdb37004-2d3f-436f-b6a8-4586ac2dfe4c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:44.534441 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fdb37004-2d3f-436f-b6a8-4586ac2dfe4c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:44.535077 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fdb37004-2d3f-436f-b6a8-4586ac2dfe4c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:44.535850 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fdb37004-2d3f-436f-b6a8-4586ac2dfe4c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:44.536299 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fdb37004-2d3f-436f-b6a8-4586ac2dfe4c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:44.536718 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fdb37004-2d3f-436f-b6a8-4586ac2dfe4c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:44.674611 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-fdb37004-2d3f-436f-b6a8-4586ac2dfe4c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:44.686183 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-fdb37004-2d3f-436f-b6a8-4586ac2dfe4c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:44.687652 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-fdb37004-2d3f-436f-b6a8-4586ac2dfe4c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:44.688802 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-fdb37004-2d3f-436f-b6a8-4586ac2dfe4c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1742 microversion: 2.20 time: 0.175938 Sep 04 01:07:44.689389 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 207/433] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:07:44 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1742 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:07:45.117240 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-a7422a0b-ef07-443e-b2f8-945bf1b47b3c tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:45.117846 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-a7422a0b-ef07-443e-b2f8-945bf1b47b3c tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:45.124120 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7422a0b-ef07-443e-b2f8-945bf1b47b3c tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:45.124592 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7422a0b-ef07-443e-b2f8-945bf1b47b3c tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:45.125026 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7422a0b-ef07-443e-b2f8-945bf1b47b3c tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:45.125727 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7422a0b-ef07-443e-b2f8-945bf1b47b3c tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:45.126170 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7422a0b-ef07-443e-b2f8-945bf1b47b3c tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:45.126589 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7422a0b-ef07-443e-b2f8-945bf1b47b3c tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:45.256777 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-a7422a0b-ef07-443e-b2f8-945bf1b47b3c tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:45.264379 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-a7422a0b-ef07-443e-b2f8-945bf1b47b3c tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5" status: 200 len: 1314 microversion: 2.1 time: 0.151770 Sep 04 01:07:45.264943 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 227/434] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:45 2022] GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 => generated 1314 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:45.303617 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-618c1507-8f0b-467d-9151-c050db892cb2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:45.304225 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-618c1507-8f0b-467d-9151-c050db892cb2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:45.310626 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-618c1507-8f0b-467d-9151-c050db892cb2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:45.311105 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-618c1507-8f0b-467d-9151-c050db892cb2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:45.311538 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-618c1507-8f0b-467d-9151-c050db892cb2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:45.312217 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-618c1507-8f0b-467d-9151-c050db892cb2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:45.312658 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-618c1507-8f0b-467d-9151-c050db892cb2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:45.313101 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-618c1507-8f0b-467d-9151-c050db892cb2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:45.381592 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-618c1507-8f0b-467d-9151-c050db892cb2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:45.389196 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-618c1507-8f0b-467d-9151-c050db892cb2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1294 microversion: 2.1 time: 0.089534 Sep 04 01:07:45.389759 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 208/435] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:45 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1294 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:45.540845 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-7b3d924e-2140-4b0d-ac87-dc8b8227a7b6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:45.541496 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-7b3d924e-2140-4b0d-ac87-dc8b8227a7b6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:45.547593 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7b3d924e-2140-4b0d-ac87-dc8b8227a7b6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:45.548081 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7b3d924e-2140-4b0d-ac87-dc8b8227a7b6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:45.548512 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7b3d924e-2140-4b0d-ac87-dc8b8227a7b6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:45.549127 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-ed593baa-44b5-4747-80b3-456a66a1346e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:45.549262 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7b3d924e-2140-4b0d-ac87-dc8b8227a7b6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:45.549670 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7b3d924e-2140-4b0d-ac87-dc8b8227a7b6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:45.549787 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-ed593baa-44b5-4747-80b3-456a66a1346e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:45.550093 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7b3d924e-2140-4b0d-ac87-dc8b8227a7b6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:45.555914 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ed593baa-44b5-4747-80b3-456a66a1346e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:45.556397 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ed593baa-44b5-4747-80b3-456a66a1346e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:45.556840 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ed593baa-44b5-4747-80b3-456a66a1346e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:45.557564 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ed593baa-44b5-4747-80b3-456a66a1346e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:45.558015 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ed593baa-44b5-4747-80b3-456a66a1346e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:45.558504 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ed593baa-44b5-4747-80b3-456a66a1346e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:45.689331 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-7b3d924e-2140-4b0d-ac87-dc8b8227a7b6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:45.695301 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-ed593baa-44b5-4747-80b3-456a66a1346e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:45.698220 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-7b3d924e-2140-4b0d-ac87-dc8b8227a7b6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1371 microversion: 2.1 time: 0.162662 Sep 04 01:07:45.698883 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 228/436] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:45 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1371 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:45.703609 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-ed593baa-44b5-4747-80b3-456a66a1346e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1706 microversion: 2.1 time: 0.158228 Sep 04 01:07:45.704167 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 209/437] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:45 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1706 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:45.719762 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-2fed145e-745c-4387-80f7-31513c552985 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:45.720484 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-2fed145e-745c-4387-80f7-31513c552985 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:45.726514 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2fed145e-745c-4387-80f7-31513c552985 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:45.726990 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2fed145e-745c-4387-80f7-31513c552985 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:45.727422 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2fed145e-745c-4387-80f7-31513c552985 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:45.728105 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2fed145e-745c-4387-80f7-31513c552985 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:45.728556 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2fed145e-745c-4387-80f7-31513c552985 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:45.728982 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2fed145e-745c-4387-80f7-31513c552985 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:45.859184 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-2fed145e-745c-4387-80f7-31513c552985 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:45.868935 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-2fed145e-745c-4387-80f7-31513c552985 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:45.870415 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-2fed145e-745c-4387-80f7-31513c552985 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:45.871578 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-2fed145e-745c-4387-80f7-31513c552985 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1742 microversion: 2.20 time: 0.155808 Sep 04 01:07:45.872139 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 229/438] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:07:45 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1742 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:07:46.290044 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-b9d9938b-d303-4840-8dca-48f0db379795 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:46.290728 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-b9d9938b-d303-4840-8dca-48f0db379795 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:46.297547 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b9d9938b-d303-4840-8dca-48f0db379795 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:46.298026 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b9d9938b-d303-4840-8dca-48f0db379795 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:46.298455 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b9d9938b-d303-4840-8dca-48f0db379795 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:46.299140 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b9d9938b-d303-4840-8dca-48f0db379795 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:46.299612 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b9d9938b-d303-4840-8dca-48f0db379795 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:46.300017 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b9d9938b-d303-4840-8dca-48f0db379795 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:46.413275 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b56c8461-ffd4-46c9-b88d-0c20fca7b58a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:46.414094 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-b56c8461-ffd4-46c9-b88d-0c20fca7b58a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:46.420731 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b56c8461-ffd4-46c9-b88d-0c20fca7b58a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:46.421259 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b56c8461-ffd4-46c9-b88d-0c20fca7b58a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:46.421696 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b56c8461-ffd4-46c9-b88d-0c20fca7b58a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:46.422581 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b56c8461-ffd4-46c9-b88d-0c20fca7b58a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:46.423030 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b56c8461-ffd4-46c9-b88d-0c20fca7b58a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:46.424127 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b56c8461-ffd4-46c9-b88d-0c20fca7b58a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:46.449310 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-b9d9938b-d303-4840-8dca-48f0db379795 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:46.465709 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-b9d9938b-d303-4840-8dca-48f0db379795 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5" status: 200 len: 1314 microversion: 2.1 time: 0.181303 Sep 04 01:07:46.471605 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 210/439] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:46 2022] GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 => generated 1314 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:46.496100 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-b56c8461-ffd4-46c9-b88d-0c20fca7b58a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:46.505188 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-b56c8461-ffd4-46c9-b88d-0c20fca7b58a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1294 microversion: 2.1 time: 0.096583 Sep 04 01:07:46.505997 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 230/440] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:46 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1294 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:07:46.524503 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-bd80c58c-b207-4e59-ba9f-7c01ad9765ae req-f9057488-cfb6-403f-a1e0-4113ae4141f9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "49be5a0b-f748-486e-a546-c07a8e8451a3", "name": "network-vif-plugged", "status": "completed", "tag": "fcbc60fd-1785-4f94-9436-a520ec06cdaf"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:07:46.530918 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-bd80c58c-b207-4e59-ba9f-7c01ad9765ae req-f9057488-cfb6-403f-a1e0-4113ae4141f9 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:46.531245 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-bd80c58c-b207-4e59-ba9f-7c01ad9765ae req-f9057488-cfb6-403f-a1e0-4113ae4141f9 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:46.531811 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-bd80c58c-b207-4e59-ba9f-7c01ad9765ae req-f9057488-cfb6-403f-a1e0-4113ae4141f9 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:46.546056 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-bd80c58c-b207-4e59-ba9f-7c01ad9765ae req-f9057488-cfb6-403f-a1e0-4113ae4141f9 service nova] Creating event network-vif-plugged:fcbc60fd-1785-4f94-9436-a520ec06cdaf for instance 49be5a0b-f748-486e-a546-c07a8e8451a3 on hv-cinder-maste Sep 04 01:07:46.551769 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-bd80c58c-b207-4e59-ba9f-7c01ad9765ae req-f9057488-cfb6-403f-a1e0-4113ae4141f9 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038612 Sep 04 01:07:46.552506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 211/441] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:07:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:07:46.721562 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-8752e732-a3bf-446f-9061-15a43dfa619d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:46.722164 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-8752e732-a3bf-446f-9061-15a43dfa619d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:46.728730 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-1ce04ddb-d774-4c84-9d9f-0606fd92877e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:46.729017 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8752e732-a3bf-446f-9061-15a43dfa619d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:46.729446 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-1ce04ddb-d774-4c84-9d9f-0606fd92877e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:46.729550 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8752e732-a3bf-446f-9061-15a43dfa619d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:46.729954 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8752e732-a3bf-446f-9061-15a43dfa619d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:46.730650 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8752e732-a3bf-446f-9061-15a43dfa619d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:46.731105 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8752e732-a3bf-446f-9061-15a43dfa619d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:46.731529 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8752e732-a3bf-446f-9061-15a43dfa619d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:46.736080 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1ce04ddb-d774-4c84-9d9f-0606fd92877e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:46.737348 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1ce04ddb-d774-4c84-9d9f-0606fd92877e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:46.737348 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1ce04ddb-d774-4c84-9d9f-0606fd92877e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:46.738060 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1ce04ddb-d774-4c84-9d9f-0606fd92877e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:46.739147 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1ce04ddb-d774-4c84-9d9f-0606fd92877e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:46.739523 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1ce04ddb-d774-4c84-9d9f-0606fd92877e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:46.867170 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-8752e732-a3bf-446f-9061-15a43dfa619d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:46.875299 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-8752e732-a3bf-446f-9061-15a43dfa619d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1371 microversion: 2.1 time: 0.157651 Sep 04 01:07:46.876116 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 231/442] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:46 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1371 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:46.888885 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-1ce04ddb-d774-4c84-9d9f-0606fd92877e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:46.897464 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-275b48fc-293c-4f92-9cac-bed1bd618866 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:46.898732 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-275b48fc-293c-4f92-9cac-bed1bd618866 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:46.898884 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-1ce04ddb-d774-4c84-9d9f-0606fd92877e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1706 microversion: 2.1 time: 0.174435 Sep 04 01:07:46.899421 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 212/443] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:46 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1706 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:07:46.904895 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-275b48fc-293c-4f92-9cac-bed1bd618866 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:46.905403 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-275b48fc-293c-4f92-9cac-bed1bd618866 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:46.905853 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-275b48fc-293c-4f92-9cac-bed1bd618866 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:46.906535 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-275b48fc-293c-4f92-9cac-bed1bd618866 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:46.906978 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-275b48fc-293c-4f92-9cac-bed1bd618866 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:46.907395 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-275b48fc-293c-4f92-9cac-bed1bd618866 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:47.036189 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-275b48fc-293c-4f92-9cac-bed1bd618866 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:47.045665 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-275b48fc-293c-4f92-9cac-bed1bd618866 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:47.047141 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-275b48fc-293c-4f92-9cac-bed1bd618866 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:47.048380 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-275b48fc-293c-4f92-9cac-bed1bd618866 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1742 microversion: 2.20 time: 0.155416 Sep 04 01:07:47.049191 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 232/444] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:07:46 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1742 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:07:47.490563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-2e088aa2-fb99-44f9-8d65-1af4b967518d tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:47.491204 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-2e088aa2-fb99-44f9-8d65-1af4b967518d tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:47.498121 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2e088aa2-fb99-44f9-8d65-1af4b967518d tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:47.498604 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2e088aa2-fb99-44f9-8d65-1af4b967518d tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:47.499036 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2e088aa2-fb99-44f9-8d65-1af4b967518d tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:47.499726 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2e088aa2-fb99-44f9-8d65-1af4b967518d tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:47.500172 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2e088aa2-fb99-44f9-8d65-1af4b967518d tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:47.500594 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2e088aa2-fb99-44f9-8d65-1af4b967518d tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:47.528329 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-0ab1a9e7-5f6c-4d07-aea9-0c7acb782370 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:47.529065 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-0ab1a9e7-5f6c-4d07-aea9-0c7acb782370 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:47.536175 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0ab1a9e7-5f6c-4d07-aea9-0c7acb782370 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:47.536654 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0ab1a9e7-5f6c-4d07-aea9-0c7acb782370 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:47.537104 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0ab1a9e7-5f6c-4d07-aea9-0c7acb782370 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:47.537789 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0ab1a9e7-5f6c-4d07-aea9-0c7acb782370 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:47.538233 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0ab1a9e7-5f6c-4d07-aea9-0c7acb782370 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:47.538653 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0ab1a9e7-5f6c-4d07-aea9-0c7acb782370 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:47.606155 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-0ab1a9e7-5f6c-4d07-aea9-0c7acb782370 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:47.614612 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-0ab1a9e7-5f6c-4d07-aea9-0c7acb782370 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1294 microversion: 2.1 time: 0.090206 Sep 04 01:07:47.615177 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 233/445] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:47 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1294 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:07:47.632394 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-2e088aa2-fb99-44f9-8d65-1af4b967518d tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:47.639968 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-2e088aa2-fb99-44f9-8d65-1af4b967518d tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5" status: 200 len: 1498 microversion: 2.1 time: 0.154010 Sep 04 01:07:47.640517 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 213/446] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:47 2022] GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 => generated 1498 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:07:47.898461 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-449ed8d6-4bd3-4250-92d4-380c12f62e6b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:47.899067 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-449ed8d6-4bd3-4250-92d4-380c12f62e6b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:47.905159 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-449ed8d6-4bd3-4250-92d4-380c12f62e6b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:47.905641 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-449ed8d6-4bd3-4250-92d4-380c12f62e6b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:47.906079 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-449ed8d6-4bd3-4250-92d4-380c12f62e6b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:47.906767 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-449ed8d6-4bd3-4250-92d4-380c12f62e6b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:47.907214 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-449ed8d6-4bd3-4250-92d4-380c12f62e6b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:47.907636 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-449ed8d6-4bd3-4250-92d4-380c12f62e6b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:47.924390 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-a8389c9e-caba-4d9f-b592-ac2056176122 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:47.926936 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-a8389c9e-caba-4d9f-b592-ac2056176122 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:47.931987 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a8389c9e-caba-4d9f-b592-ac2056176122 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:47.932467 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a8389c9e-caba-4d9f-b592-ac2056176122 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:47.933939 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a8389c9e-caba-4d9f-b592-ac2056176122 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:47.935251 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a8389c9e-caba-4d9f-b592-ac2056176122 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:47.935718 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a8389c9e-caba-4d9f-b592-ac2056176122 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:47.936142 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a8389c9e-caba-4d9f-b592-ac2056176122 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:48.092334 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-449ed8d6-4bd3-4250-92d4-380c12f62e6b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:48.111023 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-449ed8d6-4bd3-4250-92d4-380c12f62e6b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1371 microversion: 2.1 time: 0.216564 Sep 04 01:07:48.111924 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 234/447] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:47 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1371 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:48.117216 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-4b5f28f0-e953-4149-834b-4e5bc02109f5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:48.117881 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-4b5f28f0-e953-4149-834b-4e5bc02109f5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:48.124782 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4b5f28f0-e953-4149-834b-4e5bc02109f5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:48.125584 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4b5f28f0-e953-4149-834b-4e5bc02109f5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:48.126078 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4b5f28f0-e953-4149-834b-4e5bc02109f5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:48.127106 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4b5f28f0-e953-4149-834b-4e5bc02109f5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:48.127465 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4b5f28f0-e953-4149-834b-4e5bc02109f5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:48.128030 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4b5f28f0-e953-4149-834b-4e5bc02109f5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:48.187576 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell0:SELECT=10 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:48.192761 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-a8389c9e-caba-4d9f-b592-ac2056176122 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:48.202286 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-a8389c9e-caba-4d9f-b592-ac2056176122 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1706 microversion: 2.1 time: 0.282343 Sep 04 01:07:48.202933 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 214/448] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:47 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1706 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:48.272563 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-4b5f28f0-e953-4149-834b-4e5bc02109f5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:48.281401 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-4b5f28f0-e953-4149-834b-4e5bc02109f5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:48.282948 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-4b5f28f0-e953-4149-834b-4e5bc02109f5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:48.284163 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-4b5f28f0-e953-4149-834b-4e5bc02109f5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1742 microversion: 2.20 time: 0.171479 Sep 04 01:07:48.284746 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 235/449] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:07:48 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1742 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:07:48.638132 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-ba60a54d-3976-4ef4-94a0-913d2b1979e7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:48.638749 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-ba60a54d-3976-4ef4-94a0-913d2b1979e7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:48.645769 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ba60a54d-3976-4ef4-94a0-913d2b1979e7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:48.646481 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ba60a54d-3976-4ef4-94a0-913d2b1979e7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:48.646995 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ba60a54d-3976-4ef4-94a0-913d2b1979e7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:48.647856 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ba60a54d-3976-4ef4-94a0-913d2b1979e7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:48.648301 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ba60a54d-3976-4ef4-94a0-913d2b1979e7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:48.648893 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ba60a54d-3976-4ef4-94a0-913d2b1979e7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:48.764292 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-ba60a54d-3976-4ef4-94a0-913d2b1979e7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:48.777697 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-ba60a54d-3976-4ef4-94a0-913d2b1979e7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1294 microversion: 2.1 time: 0.143523 Sep 04 01:07:48.778613 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 215/450] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:48 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1294 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:48.852987 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-ccbc1498-d68f-447c-946a-8df68656d594 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:48.853547 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-ccbc1498-d68f-447c-946a-8df68656d594 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:48.861680 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ccbc1498-d68f-447c-946a-8df68656d594 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:48.865488 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ccbc1498-d68f-447c-946a-8df68656d594 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:48.865488 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ccbc1498-d68f-447c-946a-8df68656d594 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:48.865488 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ccbc1498-d68f-447c-946a-8df68656d594 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:48.865488 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ccbc1498-d68f-447c-946a-8df68656d594 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:48.865912 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ccbc1498-d68f-447c-946a-8df68656d594 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:48.890616 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-ccbc1498-d68f-447c-946a-8df68656d594 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5] Going to try to terminate instance {{(pid=121036) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 04 01:07:49.131374 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-b4163a8f-1c6f-4d81-a913-43864ed96dc0 req-6d0fc548-7d9d-485b-801b-9ef19cd75660 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2570ee6c-f335-41bc-8e67-f89a62b28134", "name": "network-vif-plugged", "status": "completed", "tag": "62f5079a-1d7d-4c5c-bdcd-0af337b6235b"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:07:49.152733 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-b4163a8f-1c6f-4d81-a913-43864ed96dc0 req-6d0fc548-7d9d-485b-801b-9ef19cd75660 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:49.155893 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-b4163a8f-1c6f-4d81-a913-43864ed96dc0 req-6d0fc548-7d9d-485b-801b-9ef19cd75660 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:49.156341 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-b4163a8f-1c6f-4d81-a913-43864ed96dc0 req-6d0fc548-7d9d-485b-801b-9ef19cd75660 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:49.166549 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:49.176493 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-b4163a8f-1c6f-4d81-a913-43864ed96dc0 req-6d0fc548-7d9d-485b-801b-9ef19cd75660 service nova] Creating event network-vif-plugged:62f5079a-1d7d-4c5c-bdcd-0af337b6235b for instance 2570ee6c-f335-41bc-8e67-f89a62b28134 on hv-cinder-maste Sep 04 01:07:49.182015 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-b4163a8f-1c6f-4d81-a913-43864ed96dc0 req-6d0fc548-7d9d-485b-801b-9ef19cd75660 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.057448 Sep 04 01:07:49.182938 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 216/451] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:07:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:07:49.192167 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-5913352a-f148-4533-9e7b-2d110f30d4fe tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:49.192167 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-5913352a-f148-4533-9e7b-2d110f30d4fe tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:49.192167 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:49.196054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5913352a-f148-4533-9e7b-2d110f30d4fe tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:49.196541 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5913352a-f148-4533-9e7b-2d110f30d4fe tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:49.196973 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5913352a-f148-4533-9e7b-2d110f30d4fe tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:49.197709 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5913352a-f148-4533-9e7b-2d110f30d4fe tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:49.198166 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5913352a-f148-4533-9e7b-2d110f30d4fe tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:49.198661 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5913352a-f148-4533-9e7b-2d110f30d4fe tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:49.308077 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-ccbc1498-d68f-447c-946a-8df68656d594 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "DELETE /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5" status: 204 len: 0 microversion: 2.1 time: 0.459325 Sep 04 01:07:49.308952 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 236/452] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 4 01:07:48 2022] DELETE /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 04 01:07:49.317446 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-a7fee38a-858f-4dcf-a9dc-7c9694c9df63 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:49.319051 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-a7fee38a-858f-4dcf-a9dc-7c9694c9df63 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:49.329049 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7fee38a-858f-4dcf-a9dc-7c9694c9df63 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:49.334093 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7fee38a-858f-4dcf-a9dc-7c9694c9df63 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:49.334093 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7fee38a-858f-4dcf-a9dc-7c9694c9df63 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:49.334093 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7fee38a-858f-4dcf-a9dc-7c9694c9df63 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:49.334093 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7fee38a-858f-4dcf-a9dc-7c9694c9df63 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:49.334481 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7fee38a-858f-4dcf-a9dc-7c9694c9df63 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:49.348196 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-5913352a-f148-4533-9e7b-2d110f30d4fe tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:49.363920 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-5913352a-f148-4533-9e7b-2d110f30d4fe tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1371 microversion: 2.1 time: 0.173669 Sep 04 01:07:49.363920 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 217/453] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:49 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1371 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:49.363920 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-b4d3a87c-d8da-4a14-b15a-8ff22013074b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:49.363920 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-b4d3a87c-d8da-4a14-b15a-8ff22013074b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:49.370103 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b4d3a87c-d8da-4a14-b15a-8ff22013074b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:49.370758 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b4d3a87c-d8da-4a14-b15a-8ff22013074b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:49.371196 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b4d3a87c-d8da-4a14-b15a-8ff22013074b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:49.371883 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b4d3a87c-d8da-4a14-b15a-8ff22013074b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:49.372336 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b4d3a87c-d8da-4a14-b15a-8ff22013074b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:49.372763 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b4d3a87c-d8da-4a14-b15a-8ff22013074b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:49.485462 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-a7fee38a-858f-4dcf-a9dc-7c9694c9df63 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:49.489178 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:49.494897 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-a7fee38a-858f-4dcf-a9dc-7c9694c9df63 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1706 microversion: 2.1 time: 0.181694 Sep 04 01:07:49.495503 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 237/454] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:49 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1706 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:49.500178 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-a2e62092-704e-402c-9843-a4776f1c552d tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:49.500784 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-a2e62092-704e-402c-9843-a4776f1c552d tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:49.507845 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a2e62092-704e-402c-9843-a4776f1c552d tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:49.508510 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a2e62092-704e-402c-9843-a4776f1c552d tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:49.509135 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a2e62092-704e-402c-9843-a4776f1c552d tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:49.510032 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a2e62092-704e-402c-9843-a4776f1c552d tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:49.510699 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a2e62092-704e-402c-9843-a4776f1c552d tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:49.511292 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a2e62092-704e-402c-9843-a4776f1c552d tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:49.524256 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-b4d3a87c-d8da-4a14-b15a-8ff22013074b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:49.531250 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-b4d3a87c-d8da-4a14-b15a-8ff22013074b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:49.532772 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-b4d3a87c-d8da-4a14-b15a-8ff22013074b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:49.533968 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-b4d3a87c-d8da-4a14-b15a-8ff22013074b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1737 microversion: 2.20 time: 0.174744 Sep 04 01:07:49.534522 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 218/455] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:07:49 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1737 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:07:49.644947 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-a2e62092-704e-402c-9843-a4776f1c552d tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:49.652535 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-a2e62092-704e-402c-9843-a4776f1c552d tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5" status: 200 len: 1504 microversion: 2.1 time: 0.156038 Sep 04 01:07:49.653105 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 238/456] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:49 2022] GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 => generated 1504 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:49.805191 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-1bb47166-04b9-456c-b02b-e6758f061928 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:49.805854 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-1bb47166-04b9-456c-b02b-e6758f061928 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:49.812893 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1bb47166-04b9-456c-b02b-e6758f061928 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:49.813400 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1bb47166-04b9-456c-b02b-e6758f061928 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:49.813833 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1bb47166-04b9-456c-b02b-e6758f061928 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:49.814513 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1bb47166-04b9-456c-b02b-e6758f061928 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:49.814960 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1bb47166-04b9-456c-b02b-e6758f061928 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:49.815382 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1bb47166-04b9-456c-b02b-e6758f061928 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:49.878846 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-1bb47166-04b9-456c-b02b-e6758f061928 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:49.887063 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-1bb47166-04b9-456c-b02b-e6758f061928 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1302 microversion: 2.1 time: 0.086171 Sep 04 01:07:49.887867 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 219/457] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:49 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1302 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:50.383640 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b9b4d8e7-0521-44c1-85d9-8485b80af0e1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:50.384245 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-b9b4d8e7-0521-44c1-85d9-8485b80af0e1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:50.391321 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b9b4d8e7-0521-44c1-85d9-8485b80af0e1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:50.391801 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b9b4d8e7-0521-44c1-85d9-8485b80af0e1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:50.392236 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b9b4d8e7-0521-44c1-85d9-8485b80af0e1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:50.392920 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b9b4d8e7-0521-44c1-85d9-8485b80af0e1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:50.393396 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b9b4d8e7-0521-44c1-85d9-8485b80af0e1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:50.393819 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b9b4d8e7-0521-44c1-85d9-8485b80af0e1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:50.519863 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-fa5dab94-f2e4-4064-b700-9c99ef4f586d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:50.520380 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-fa5dab94-f2e4-4064-b700-9c99ef4f586d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:50.526773 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fa5dab94-f2e4-4064-b700-9c99ef4f586d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:50.527251 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fa5dab94-f2e4-4064-b700-9c99ef4f586d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:50.527676 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fa5dab94-f2e4-4064-b700-9c99ef4f586d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:50.528359 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fa5dab94-f2e4-4064-b700-9c99ef4f586d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:50.528802 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fa5dab94-f2e4-4064-b700-9c99ef4f586d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:50.529255 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fa5dab94-f2e4-4064-b700-9c99ef4f586d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:50.532678 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-b9b4d8e7-0521-44c1-85d9-8485b80af0e1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:50.540969 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-b9b4d8e7-0521-44c1-85d9-8485b80af0e1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1371 microversion: 2.1 time: 0.161216 Sep 04 01:07:50.541568 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 239/458] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:50 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1371 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:50.560301 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-61839679-6e25-486b-8ba6-9904d84a3890 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:50.561158 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-61839679-6e25-486b-8ba6-9904d84a3890 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:50.567342 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-61839679-6e25-486b-8ba6-9904d84a3890 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:50.567965 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-61839679-6e25-486b-8ba6-9904d84a3890 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:50.568410 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-61839679-6e25-486b-8ba6-9904d84a3890 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:50.569136 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-61839679-6e25-486b-8ba6-9904d84a3890 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:50.569593 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-61839679-6e25-486b-8ba6-9904d84a3890 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:50.570137 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-61839679-6e25-486b-8ba6-9904d84a3890 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:50.666376 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-fa5dab94-f2e4-4064-b700-9c99ef4f586d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:50.675100 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-fa5dab94-f2e4-4064-b700-9c99ef4f586d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1706 microversion: 2.1 time: 0.159112 Sep 04 01:07:50.676012 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 220/459] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:50 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1706 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:07:50.680305 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-c67a67ca-af73-47a2-8aa8-b5aa83a39a4e tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:50.680900 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-c67a67ca-af73-47a2-8aa8-b5aa83a39a4e tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:50.686732 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c67a67ca-af73-47a2-8aa8-b5aa83a39a4e tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:50.687200 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c67a67ca-af73-47a2-8aa8-b5aa83a39a4e tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:50.687624 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c67a67ca-af73-47a2-8aa8-b5aa83a39a4e tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:50.688302 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c67a67ca-af73-47a2-8aa8-b5aa83a39a4e tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:50.688747 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c67a67ca-af73-47a2-8aa8-b5aa83a39a4e tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:50.689216 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c67a67ca-af73-47a2-8aa8-b5aa83a39a4e tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:50.712828 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-61839679-6e25-486b-8ba6-9904d84a3890 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:50.721034 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-61839679-6e25-486b-8ba6-9904d84a3890 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:50.722492 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-61839679-6e25-486b-8ba6-9904d84a3890 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:50.723636 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-61839679-6e25-486b-8ba6-9904d84a3890 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1737 microversion: 2.20 time: 0.167233 Sep 04 01:07:50.724275 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 240/460] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:07:50 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1737 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:07:50.839159 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-c67a67ca-af73-47a2-8aa8-b5aa83a39a4e tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:50.846802 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-c67a67ca-af73-47a2-8aa8-b5aa83a39a4e tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5" status: 200 len: 1504 microversion: 2.1 time: 0.169947 Sep 04 01:07:50.847493 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 221/461] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:50 2022] GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 => generated 1504 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:50.910908 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-f31cf073-5a8c-49e8-8f65-17a74eec6b21 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:50.912026 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-f31cf073-5a8c-49e8-8f65-17a74eec6b21 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:50.918032 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f31cf073-5a8c-49e8-8f65-17a74eec6b21 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:50.918604 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f31cf073-5a8c-49e8-8f65-17a74eec6b21 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:50.918981 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f31cf073-5a8c-49e8-8f65-17a74eec6b21 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:50.919664 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f31cf073-5a8c-49e8-8f65-17a74eec6b21 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:50.920125 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f31cf073-5a8c-49e8-8f65-17a74eec6b21 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:50.920531 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f31cf073-5a8c-49e8-8f65-17a74eec6b21 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:50.985714 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-f31cf073-5a8c-49e8-8f65-17a74eec6b21 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:50.994004 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-f31cf073-5a8c-49e8-8f65-17a74eec6b21 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1302 microversion: 2.1 time: 0.086960 Sep 04 01:07:50.994568 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 241/462] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:50 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1302 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:51.563675 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-ec0ffdba-5f47-4b10-b711-9aa397813357 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:51.564285 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-ec0ffdba-5f47-4b10-b711-9aa397813357 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:51.570557 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ec0ffdba-5f47-4b10-b711-9aa397813357 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:51.571046 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ec0ffdba-5f47-4b10-b711-9aa397813357 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:51.571481 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ec0ffdba-5f47-4b10-b711-9aa397813357 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:51.572171 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ec0ffdba-5f47-4b10-b711-9aa397813357 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:51.572623 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ec0ffdba-5f47-4b10-b711-9aa397813357 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:51.573046 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ec0ffdba-5f47-4b10-b711-9aa397813357 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:51.698506 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-c6d803e0-f42a-4161-8327-c6761e8a15a5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:51.699119 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-c6d803e0-f42a-4161-8327-c6761e8a15a5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:51.705700 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c6d803e0-f42a-4161-8327-c6761e8a15a5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:51.706387 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c6d803e0-f42a-4161-8327-c6761e8a15a5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:51.706899 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c6d803e0-f42a-4161-8327-c6761e8a15a5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:51.707698 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c6d803e0-f42a-4161-8327-c6761e8a15a5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:51.708905 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c6d803e0-f42a-4161-8327-c6761e8a15a5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:51.709533 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c6d803e0-f42a-4161-8327-c6761e8a15a5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:51.709645 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-ec0ffdba-5f47-4b10-b711-9aa397813357 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:51.717762 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-ec0ffdba-5f47-4b10-b711-9aa397813357 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1371 microversion: 2.1 time: 0.157851 Sep 04 01:07:51.718350 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 222/463] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:51 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1371 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:51.749958 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-d4d81504-016c-4f77-be22-a210b6ab0aed tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:51.750437 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-d4d81504-016c-4f77-be22-a210b6ab0aed tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:51.757232 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d4d81504-016c-4f77-be22-a210b6ab0aed tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:51.757726 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d4d81504-016c-4f77-be22-a210b6ab0aed tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:51.758154 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d4d81504-016c-4f77-be22-a210b6ab0aed tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:51.758847 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d4d81504-016c-4f77-be22-a210b6ab0aed tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:51.759292 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d4d81504-016c-4f77-be22-a210b6ab0aed tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:51.759713 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d4d81504-016c-4f77-be22-a210b6ab0aed tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:51.785119 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell0:SELECT=2 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:51.847742 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-c6d803e0-f42a-4161-8327-c6761e8a15a5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:51.856065 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-c6d803e0-f42a-4161-8327-c6761e8a15a5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1706 microversion: 2.1 time: 0.161349 Sep 04 01:07:51.862223 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 242/464] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:51 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1706 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:07:51.873563 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-18939300-2f1c-448b-a763-4f17d437469a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:51.874235 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-18939300-2f1c-448b-a763-4f17d437469a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:51.881224 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-18939300-2f1c-448b-a763-4f17d437469a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:51.881708 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-18939300-2f1c-448b-a763-4f17d437469a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:51.882149 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-18939300-2f1c-448b-a763-4f17d437469a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:51.882836 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-18939300-2f1c-448b-a763-4f17d437469a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:51.883295 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-18939300-2f1c-448b-a763-4f17d437469a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:51.883705 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-18939300-2f1c-448b-a763-4f17d437469a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:51.897126 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-d4d81504-016c-4f77-be22-a210b6ab0aed tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:51.905172 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-d4d81504-016c-4f77-be22-a210b6ab0aed tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:51.906626 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-d4d81504-016c-4f77-be22-a210b6ab0aed tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:51.907778 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-d4d81504-016c-4f77-be22-a210b6ab0aed tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1737 microversion: 2.20 time: 0.161853 Sep 04 01:07:51.908529 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 223/465] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:07:51 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1737 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:07:52.013578 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-18939300-2f1c-448b-a763-4f17d437469a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:52.017662 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-aa42fa0e-8a33-4834-9a6b-3dd63f9ae8d7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:52.018269 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-aa42fa0e-8a33-4834-9a6b-3dd63f9ae8d7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:52.021209 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-18939300-2f1c-448b-a763-4f17d437469a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5" status: 200 len: 1504 microversion: 2.1 time: 0.154603 Sep 04 01:07:52.021778 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 243/466] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:51 2022] GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 => generated 1504 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:07:52.024636 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-aa42fa0e-8a33-4834-9a6b-3dd63f9ae8d7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:52.025217 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-aa42fa0e-8a33-4834-9a6b-3dd63f9ae8d7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:52.025653 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-aa42fa0e-8a33-4834-9a6b-3dd63f9ae8d7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:52.026342 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-aa42fa0e-8a33-4834-9a6b-3dd63f9ae8d7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:52.026793 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-aa42fa0e-8a33-4834-9a6b-3dd63f9ae8d7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:52.027216 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-aa42fa0e-8a33-4834-9a6b-3dd63f9ae8d7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:52.049912 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell0:SELECT=24 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:52.099430 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-aa42fa0e-8a33-4834-9a6b-3dd63f9ae8d7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:52.107367 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-aa42fa0e-8a33-4834-9a6b-3dd63f9ae8d7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1302 microversion: 2.1 time: 0.093445 Sep 04 01:07:52.107950 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 224/467] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:52 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1302 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:07:52.740513 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-0355714b-fcfc-4f1f-9934-11210603f202 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:52.741132 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-0355714b-fcfc-4f1f-9934-11210603f202 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:52.747290 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0355714b-fcfc-4f1f-9934-11210603f202 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:52.747769 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0355714b-fcfc-4f1f-9934-11210603f202 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:52.748197 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0355714b-fcfc-4f1f-9934-11210603f202 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:52.748881 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0355714b-fcfc-4f1f-9934-11210603f202 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:52.749361 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0355714b-fcfc-4f1f-9934-11210603f202 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:52.749781 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0355714b-fcfc-4f1f-9934-11210603f202 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:52.879830 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-44a7f2be-35d5-4a29-bbc6-b1ed163e0b41 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:52.880588 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-44a7f2be-35d5-4a29-bbc6-b1ed163e0b41 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:52.884822 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-0355714b-fcfc-4f1f-9934-11210603f202 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:52.887118 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-44a7f2be-35d5-4a29-bbc6-b1ed163e0b41 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:52.887599 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-44a7f2be-35d5-4a29-bbc6-b1ed163e0b41 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:52.888033 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-44a7f2be-35d5-4a29-bbc6-b1ed163e0b41 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:52.888767 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-44a7f2be-35d5-4a29-bbc6-b1ed163e0b41 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:52.889246 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-44a7f2be-35d5-4a29-bbc6-b1ed163e0b41 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:52.889685 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-44a7f2be-35d5-4a29-bbc6-b1ed163e0b41 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:52.892920 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-0355714b-fcfc-4f1f-9934-11210603f202 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1371 microversion: 2.1 time: 0.156333 Sep 04 01:07:52.893706 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 244/468] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:52 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1371 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:52.933660 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-032e300b-84a6-45f2-b20b-cfbd6899cb84 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:52.934285 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-032e300b-84a6-45f2-b20b-cfbd6899cb84 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:52.940223 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-032e300b-84a6-45f2-b20b-cfbd6899cb84 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:52.940699 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-032e300b-84a6-45f2-b20b-cfbd6899cb84 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:52.941156 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-032e300b-84a6-45f2-b20b-cfbd6899cb84 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:52.941849 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-032e300b-84a6-45f2-b20b-cfbd6899cb84 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:52.942288 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-032e300b-84a6-45f2-b20b-cfbd6899cb84 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:52.942729 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-032e300b-84a6-45f2-b20b-cfbd6899cb84 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:53.025156 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-44a7f2be-35d5-4a29-bbc6-b1ed163e0b41 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:53.033592 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-44a7f2be-35d5-4a29-bbc6-b1ed163e0b41 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1706 microversion: 2.1 time: 0.157601 Sep 04 01:07:53.034173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 225/469] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:52 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1706 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:07:53.043697 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-d75b4ee3-55ea-4f71-97c1-a0491af40788 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:53.044452 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-d75b4ee3-55ea-4f71-97c1-a0491af40788 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:53.050592 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d75b4ee3-55ea-4f71-97c1-a0491af40788 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:53.051150 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d75b4ee3-55ea-4f71-97c1-a0491af40788 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:53.051661 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d75b4ee3-55ea-4f71-97c1-a0491af40788 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:53.052813 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d75b4ee3-55ea-4f71-97c1-a0491af40788 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:53.053358 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d75b4ee3-55ea-4f71-97c1-a0491af40788 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:53.053800 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d75b4ee3-55ea-4f71-97c1-a0491af40788 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:53.076150 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-032e300b-84a6-45f2-b20b-cfbd6899cb84 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:53.084988 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-032e300b-84a6-45f2-b20b-cfbd6899cb84 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:53.086558 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-032e300b-84a6-45f2-b20b-cfbd6899cb84 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:53.087826 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-032e300b-84a6-45f2-b20b-cfbd6899cb84 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1737 microversion: 2.20 time: 0.158007 Sep 04 01:07:53.088277 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 245/470] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:07:52 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1737 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:07:53.133457 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-7b97b797-54b6-4712-94d3-460bdb2419b8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:53.134072 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-7b97b797-54b6-4712-94d3-460bdb2419b8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:53.140045 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7b97b797-54b6-4712-94d3-460bdb2419b8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:53.140535 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7b97b797-54b6-4712-94d3-460bdb2419b8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:53.145317 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7b97b797-54b6-4712-94d3-460bdb2419b8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:53.145317 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7b97b797-54b6-4712-94d3-460bdb2419b8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:53.145317 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7b97b797-54b6-4712-94d3-460bdb2419b8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:53.145317 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7b97b797-54b6-4712-94d3-460bdb2419b8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:53.182802 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-d75b4ee3-55ea-4f71-97c1-a0491af40788 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:53.191034 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-d75b4ee3-55ea-4f71-97c1-a0491af40788 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5" status: 200 len: 1504 microversion: 2.1 time: 0.150721 Sep 04 01:07:53.191364 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 226/471] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:53 2022] GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 => generated 1504 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:07:53.204220 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-7b97b797-54b6-4712-94d3-460bdb2419b8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:53.212875 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-7b97b797-54b6-4712-94d3-460bdb2419b8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1302 microversion: 2.1 time: 0.083998 Sep 04 01:07:53.216990 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 246/472] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:53 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1302 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:07:53.916645 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-fee88bc8-0a24-476c-a84f-6338d6fe337f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:53.917293 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-fee88bc8-0a24-476c-a84f-6338d6fe337f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:53.930123 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fee88bc8-0a24-476c-a84f-6338d6fe337f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:53.930744 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fee88bc8-0a24-476c-a84f-6338d6fe337f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:53.931433 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fee88bc8-0a24-476c-a84f-6338d6fe337f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:53.932480 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fee88bc8-0a24-476c-a84f-6338d6fe337f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:53.933402 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fee88bc8-0a24-476c-a84f-6338d6fe337f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:53.934114 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fee88bc8-0a24-476c-a84f-6338d6fe337f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:54.057810 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-0d3e569d-7fed-49f6-af22-755a61dc5863 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:54.058396 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-0d3e569d-7fed-49f6-af22-755a61dc5863 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:54.074847 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0d3e569d-7fed-49f6-af22-755a61dc5863 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:54.075336 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0d3e569d-7fed-49f6-af22-755a61dc5863 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:54.075770 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0d3e569d-7fed-49f6-af22-755a61dc5863 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:54.076478 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0d3e569d-7fed-49f6-af22-755a61dc5863 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:54.076912 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0d3e569d-7fed-49f6-af22-755a61dc5863 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:54.077358 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0d3e569d-7fed-49f6-af22-755a61dc5863 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:54.197792 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-fee88bc8-0a24-476c-a84f-6338d6fe337f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:54.206127 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-fee88bc8-0a24-476c-a84f-6338d6fe337f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1371 microversion: 2.1 time: 0.293675 Sep 04 01:07:54.206692 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 227/473] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:53 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1371 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:54.212329 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-11128d58-5724-4a96-affa-e107e3d25c20 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:54.213042 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-11128d58-5724-4a96-affa-e107e3d25c20 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:54.220000 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-11128d58-5724-4a96-affa-e107e3d25c20 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:54.220705 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-11128d58-5724-4a96-affa-e107e3d25c20 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:54.221312 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-11128d58-5724-4a96-affa-e107e3d25c20 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:54.222146 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-11128d58-5724-4a96-affa-e107e3d25c20 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:54.222718 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-11128d58-5724-4a96-affa-e107e3d25c20 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:54.223155 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-11128d58-5724-4a96-affa-e107e3d25c20 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:54.237550 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-0d3e569d-7fed-49f6-af22-755a61dc5863 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:54.245800 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-0d3e569d-7fed-49f6-af22-755a61dc5863 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1706 microversion: 2.1 time: 0.191887 Sep 04 01:07:54.246361 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 247/474] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:54 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1706 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:54.250756 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-7ea0fee9-7f04-4d26-a8f6-ff41478b3e4a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:54.251733 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-7ea0fee9-7f04-4d26-a8f6-ff41478b3e4a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:54.257577 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7ea0fee9-7f04-4d26-a8f6-ff41478b3e4a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:54.258064 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7ea0fee9-7f04-4d26-a8f6-ff41478b3e4a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:54.258505 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7ea0fee9-7f04-4d26-a8f6-ff41478b3e4a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:54.259254 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7ea0fee9-7f04-4d26-a8f6-ff41478b3e4a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:54.259702 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7ea0fee9-7f04-4d26-a8f6-ff41478b3e4a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:54.260114 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7ea0fee9-7f04-4d26-a8f6-ff41478b3e4a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:54.375395 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-11128d58-5724-4a96-affa-e107e3d25c20 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:54.392852 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-11128d58-5724-4a96-affa-e107e3d25c20 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:54.399824 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-11128d58-5724-4a96-affa-e107e3d25c20 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:54.402472 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-11128d58-5724-4a96-affa-e107e3d25c20 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1737 microversion: 2.20 time: 0.192542 Sep 04 01:07:54.413260 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 228/475] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:07:54 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1737 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:07:54.413260 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-c1cf4afe-fd61-42c9-ac10-50374e703b17 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:54.413260 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-c1cf4afe-fd61-42c9-ac10-50374e703b17 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:54.421306 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-7ea0fee9-7f04-4d26-a8f6-ff41478b3e4a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:54.423180 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c1cf4afe-fd61-42c9-ac10-50374e703b17 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:54.426965 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c1cf4afe-fd61-42c9-ac10-50374e703b17 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:54.426965 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c1cf4afe-fd61-42c9-ac10-50374e703b17 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:54.426965 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c1cf4afe-fd61-42c9-ac10-50374e703b17 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:54.426965 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c1cf4afe-fd61-42c9-ac10-50374e703b17 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:54.427493 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c1cf4afe-fd61-42c9-ac10-50374e703b17 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:54.436881 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-7ea0fee9-7f04-4d26-a8f6-ff41478b3e4a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5" status: 200 len: 1504 microversion: 2.1 time: 0.189441 Sep 04 01:07:54.438001 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 248/476] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:54 2022] GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 => generated 1504 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:54.547584 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-c1cf4afe-fd61-42c9-ac10-50374e703b17 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:54.563871 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-c1cf4afe-fd61-42c9-ac10-50374e703b17 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1300 microversion: 2.1 time: 0.159745 Sep 04 01:07:54.564569 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 229/477] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:54 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1300 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:55.229764 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-3af10ecb-06d7-49b8-973f-ae55e0c361e4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:55.230542 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-3af10ecb-06d7-49b8-973f-ae55e0c361e4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:55.244663 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3af10ecb-06d7-49b8-973f-ae55e0c361e4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:55.245464 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3af10ecb-06d7-49b8-973f-ae55e0c361e4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:55.246066 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3af10ecb-06d7-49b8-973f-ae55e0c361e4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:55.246963 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3af10ecb-06d7-49b8-973f-ae55e0c361e4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:55.247588 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3af10ecb-06d7-49b8-973f-ae55e0c361e4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:55.248162 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3af10ecb-06d7-49b8-973f-ae55e0c361e4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:55.272323 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-a9e1d3ad-6de1-43f1-b862-83158a526b1c tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:55.273595 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-a9e1d3ad-6de1-43f1-b862-83158a526b1c tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:55.280410 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a9e1d3ad-6de1-43f1-b862-83158a526b1c tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:55.281367 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a9e1d3ad-6de1-43f1-b862-83158a526b1c tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:55.282165 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a9e1d3ad-6de1-43f1-b862-83158a526b1c tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:55.289109 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a9e1d3ad-6de1-43f1-b862-83158a526b1c tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:55.290095 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a9e1d3ad-6de1-43f1-b862-83158a526b1c tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:55.290690 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a9e1d3ad-6de1-43f1-b862-83158a526b1c tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:55.548997 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-a9e1d3ad-6de1-43f1-b862-83158a526b1c tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:55.553472 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-3af10ecb-06d7-49b8-973f-ae55e0c361e4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:55.559681 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-a9e1d3ad-6de1-43f1-b862-83158a526b1c tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1710 microversion: 2.1 time: 0.291638 Sep 04 01:07:55.560291 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 230/478] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:55 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1710 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:07:55.566322 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-3af10ecb-06d7-49b8-973f-ae55e0c361e4 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1371 microversion: 2.1 time: 0.340389 Sep 04 01:07:55.575208 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-766b2874-44d1-4dc2-898a-8decac9e0a29 req-a7ce44fe-d15e-4a21-a9bc-3807fa3a11b7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5", "name": "network-vif-deleted", "tag": "b8947ab0-23c1-4eaa-93c4-9c1cc435d43e"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:07:55.585243 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 249/479] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:55 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1371 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:55.597730 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-766b2874-44d1-4dc2-898a-8decac9e0a29 req-a7ce44fe-d15e-4a21-a9bc-3807fa3a11b7 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:55.601677 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-766b2874-44d1-4dc2-898a-8decac9e0a29 req-a7ce44fe-d15e-4a21-a9bc-3807fa3a11b7 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:55.602638 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-766b2874-44d1-4dc2-898a-8decac9e0a29 req-a7ce44fe-d15e-4a21-a9bc-3807fa3a11b7 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:55.608830 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-a1c65ac1-c1de-4561-ad24-3f480cdfbc24 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:55.610992 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-a1c65ac1-c1de-4561-ad24-3f480cdfbc24 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:55.619017 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a1c65ac1-c1de-4561-ad24-3f480cdfbc24 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:55.619980 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a1c65ac1-c1de-4561-ad24-3f480cdfbc24 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:55.621004 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a1c65ac1-c1de-4561-ad24-3f480cdfbc24 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:55.621852 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a1c65ac1-c1de-4561-ad24-3f480cdfbc24 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:55.622411 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a1c65ac1-c1de-4561-ad24-3f480cdfbc24 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:55.623034 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a1c65ac1-c1de-4561-ad24-3f480cdfbc24 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:55.631689 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-766b2874-44d1-4dc2-898a-8decac9e0a29 req-a7ce44fe-d15e-4a21-a9bc-3807fa3a11b7 service nova] Creating event network-vif-deleted:b8947ab0-23c1-4eaa-93c4-9c1cc435d43e for instance e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 on hv-cinder-maste Sep 04 01:07:55.637428 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-766b2874-44d1-4dc2-898a-8decac9e0a29 req-a7ce44fe-d15e-4a21-a9bc-3807fa3a11b7 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.075427 Sep 04 01:07:55.638184 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 231/480] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:07:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:07:55.649368 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-2a32da4c-8f95-4889-b058-540a782ba8c5 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:55.653657 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-2a32da4c-8f95-4889-b058-540a782ba8c5 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:55.666354 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2a32da4c-8f95-4889-b058-540a782ba8c5 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:55.666850 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2a32da4c-8f95-4889-b058-540a782ba8c5 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:55.667287 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2a32da4c-8f95-4889-b058-540a782ba8c5 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:55.668020 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2a32da4c-8f95-4889-b058-540a782ba8c5 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:55.668802 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2a32da4c-8f95-4889-b058-540a782ba8c5 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:55.669263 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2a32da4c-8f95-4889-b058-540a782ba8c5 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:55.813136 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-2a32da4c-8f95-4889-b058-540a782ba8c5 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:55.841716 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-2a32da4c-8f95-4889-b058-540a782ba8c5 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5" status: 200 len: 1462 microversion: 2.1 time: 0.202043 Sep 04 01:07:55.842775 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 232/481] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:55 2022] GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 => generated 1462 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:55.848521 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-c69e22b8-1290-4320-99e1-3f06cd67923c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:55.849792 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-c69e22b8-1290-4320-99e1-3f06cd67923c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:55.857033 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c69e22b8-1290-4320-99e1-3f06cd67923c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:55.857688 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c69e22b8-1290-4320-99e1-3f06cd67923c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:55.858652 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c69e22b8-1290-4320-99e1-3f06cd67923c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:55.859905 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c69e22b8-1290-4320-99e1-3f06cd67923c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:55.860464 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c69e22b8-1290-4320-99e1-3f06cd67923c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:55.861020 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c69e22b8-1290-4320-99e1-3f06cd67923c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:55.937612 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-a1c65ac1-c1de-4561-ad24-3f480cdfbc24 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:55.956501 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-a1c65ac1-c1de-4561-ad24-3f480cdfbc24 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:55.956703 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-a1c65ac1-c1de-4561-ad24-3f480cdfbc24 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:55.958455 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-a1c65ac1-c1de-4561-ad24-3f480cdfbc24 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1737 microversion: 2.20 time: 0.353725 Sep 04 01:07:55.959521 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 250/482] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:07:55 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1737 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:07:56.072305 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-c69e22b8-1290-4320-99e1-3f06cd67923c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:56.081695 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-c69e22b8-1290-4320-99e1-3f06cd67923c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1368 microversion: 2.1 time: 0.237761 Sep 04 01:07:56.093905 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 233/483] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:55 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1368 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:56.597536 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-90661fe9-ee1a-4a35-885a-d8c856890c8a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:56.599155 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-90661fe9-ee1a-4a35-885a-d8c856890c8a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:56.606072 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-90661fe9-ee1a-4a35-885a-d8c856890c8a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:56.606915 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-90661fe9-ee1a-4a35-885a-d8c856890c8a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:56.607920 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-90661fe9-ee1a-4a35-885a-d8c856890c8a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:56.608892 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-90661fe9-ee1a-4a35-885a-d8c856890c8a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:56.609728 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-90661fe9-ee1a-4a35-885a-d8c856890c8a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:56.610393 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-90661fe9-ee1a-4a35-885a-d8c856890c8a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:56.819561 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-90661fe9-ee1a-4a35-885a-d8c856890c8a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:56.827448 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-90661fe9-ee1a-4a35-885a-d8c856890c8a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1371 microversion: 2.1 time: 0.233889 Sep 04 01:07:56.828165 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 251/484] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:56 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1371 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:56.870265 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-661983e2-8096-40be-83b5-0dfb73115c40 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:56.871648 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-661983e2-8096-40be-83b5-0dfb73115c40 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:56.879402 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-661983e2-8096-40be-83b5-0dfb73115c40 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:56.880615 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-661983e2-8096-40be-83b5-0dfb73115c40 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:56.881316 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-661983e2-8096-40be-83b5-0dfb73115c40 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:56.882454 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-661983e2-8096-40be-83b5-0dfb73115c40 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:56.883113 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-661983e2-8096-40be-83b5-0dfb73115c40 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:56.883761 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-661983e2-8096-40be-83b5-0dfb73115c40 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:56.887428 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=100 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:56.954693 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-661983e2-8096-40be-83b5-0dfb73115c40 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:56.966753 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-661983e2-8096-40be-83b5-0dfb73115c40 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5" status: 200 len: 1462 microversion: 2.1 time: 0.100554 Sep 04 01:07:56.967644 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 234/485] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:56 2022] GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 => generated 1462 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:56.991037 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-cdb17653-9c53-497c-bdab-2a6f5f0c9751 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:56.992145 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-cdb17653-9c53-497c-bdab-2a6f5f0c9751 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:57.010093 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cdb17653-9c53-497c-bdab-2a6f5f0c9751 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:57.010569 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cdb17653-9c53-497c-bdab-2a6f5f0c9751 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:57.011004 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cdb17653-9c53-497c-bdab-2a6f5f0c9751 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:57.011713 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cdb17653-9c53-497c-bdab-2a6f5f0c9751 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:57.012162 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cdb17653-9c53-497c-bdab-2a6f5f0c9751 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:57.012805 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cdb17653-9c53-497c-bdab-2a6f5f0c9751 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:57.110239 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-21fbdb99-b517-4a13-8e54-55c6232a1261 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:57.111255 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-21fbdb99-b517-4a13-8e54-55c6232a1261 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:57.118632 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-21fbdb99-b517-4a13-8e54-55c6232a1261 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:57.119113 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-21fbdb99-b517-4a13-8e54-55c6232a1261 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:57.121483 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-21fbdb99-b517-4a13-8e54-55c6232a1261 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:57.122375 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-21fbdb99-b517-4a13-8e54-55c6232a1261 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:57.123035 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-21fbdb99-b517-4a13-8e54-55c6232a1261 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:57.123603 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-21fbdb99-b517-4a13-8e54-55c6232a1261 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:57.191836 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-cdb17653-9c53-497c-bdab-2a6f5f0c9751 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:57.202610 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-cdb17653-9c53-497c-bdab-2a6f5f0c9751 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:57.204106 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-cdb17653-9c53-497c-bdab-2a6f5f0c9751 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:57.205392 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-cdb17653-9c53-497c-bdab-2a6f5f0c9751 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1737 microversion: 2.20 time: 0.218965 Sep 04 01:07:57.209701 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 252/486] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:07:56 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1737 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:07:57.305359 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-21fbdb99-b517-4a13-8e54-55c6232a1261 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:57.313521 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-21fbdb99-b517-4a13-8e54-55c6232a1261 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1368 microversion: 2.1 time: 0.209952 Sep 04 01:07:57.314102 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 235/487] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:57 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1368 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:07:57.851094 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-10b2b209-b764-4704-9ab4-23b6f9d794a5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:57.853409 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-10b2b209-b764-4704-9ab4-23b6f9d794a5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:57.858901 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-10b2b209-b764-4704-9ab4-23b6f9d794a5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:57.859647 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-10b2b209-b764-4704-9ab4-23b6f9d794a5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:57.860082 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-10b2b209-b764-4704-9ab4-23b6f9d794a5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:57.860782 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-10b2b209-b764-4704-9ab4-23b6f9d794a5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:57.861271 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-10b2b209-b764-4704-9ab4-23b6f9d794a5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:57.861695 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-10b2b209-b764-4704-9ab4-23b6f9d794a5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:57.995487 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-23e51bed-3e16-436e-9727-e0cbe1b98865 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:57.996174 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-23e51bed-3e16-436e-9727-e0cbe1b98865 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:58.002874 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-23e51bed-3e16-436e-9727-e0cbe1b98865 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:58.003347 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-23e51bed-3e16-436e-9727-e0cbe1b98865 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:58.003779 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-23e51bed-3e16-436e-9727-e0cbe1b98865 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:58.004472 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-23e51bed-3e16-436e-9727-e0cbe1b98865 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:58.005692 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-23e51bed-3e16-436e-9727-e0cbe1b98865 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:58.007448 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-23e51bed-3e16-436e-9727-e0cbe1b98865 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:58.021690 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-10b2b209-b764-4704-9ab4-23b6f9d794a5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:58.029676 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-10b2b209-b764-4704-9ab4-23b6f9d794a5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1371 microversion: 2.1 time: 0.182568 Sep 04 01:07:58.030282 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 253/488] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:57 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1371 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:58.076526 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-23e51bed-3e16-436e-9727-e0cbe1b98865 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:58.084462 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-23e51bed-3e16-436e-9727-e0cbe1b98865 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5" status: 200 len: 1462 microversion: 2.1 time: 0.092809 Sep 04 01:07:58.103429 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 236/489] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:57 2022] GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 => generated 1462 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:58.235670 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-e21e0f2a-2945-4f48-a428-0b0448825156 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:58.236310 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-e21e0f2a-2945-4f48-a428-0b0448825156 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:58.242846 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e21e0f2a-2945-4f48-a428-0b0448825156 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:58.243326 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e21e0f2a-2945-4f48-a428-0b0448825156 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:58.243756 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e21e0f2a-2945-4f48-a428-0b0448825156 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:58.244440 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e21e0f2a-2945-4f48-a428-0b0448825156 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:58.245042 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e21e0f2a-2945-4f48-a428-0b0448825156 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:58.245492 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e21e0f2a-2945-4f48-a428-0b0448825156 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:58.264509 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=100 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:07:58.336928 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-58633da3-76a4-4c11-926b-d86aeda90317 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:58.337560 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-58633da3-76a4-4c11-926b-d86aeda90317 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:58.345758 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-58633da3-76a4-4c11-926b-d86aeda90317 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:58.346248 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-58633da3-76a4-4c11-926b-d86aeda90317 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:58.346681 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-58633da3-76a4-4c11-926b-d86aeda90317 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:58.347379 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-58633da3-76a4-4c11-926b-d86aeda90317 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:58.347827 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-58633da3-76a4-4c11-926b-d86aeda90317 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:58.349294 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-58633da3-76a4-4c11-926b-d86aeda90317 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:58.375007 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e21e0f2a-2945-4f48-a428-0b0448825156 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:58.383252 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e21e0f2a-2945-4f48-a428-0b0448825156 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:58.384796 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e21e0f2a-2945-4f48-a428-0b0448825156 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:58.385976 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-e21e0f2a-2945-4f48-a428-0b0448825156 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1737 microversion: 2.20 time: 0.154600 Sep 04 01:07:58.386532 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 254/490] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:07:58 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1737 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:07:58.492734 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-58633da3-76a4-4c11-926b-d86aeda90317 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:58.504402 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-58633da3-76a4-4c11-926b-d86aeda90317 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1368 microversion: 2.1 time: 0.171147 Sep 04 01:07:58.505046 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 237/491] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:58 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1368 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:59.054566 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-105dc648-2ae9-4948-a66b-c79bfc9c573f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:59.055206 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-105dc648-2ae9-4948-a66b-c79bfc9c573f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:59.063138 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-105dc648-2ae9-4948-a66b-c79bfc9c573f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:59.064067 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-105dc648-2ae9-4948-a66b-c79bfc9c573f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:59.064681 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-105dc648-2ae9-4948-a66b-c79bfc9c573f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:59.065678 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-105dc648-2ae9-4948-a66b-c79bfc9c573f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:59.066347 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-105dc648-2ae9-4948-a66b-c79bfc9c573f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:59.067164 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-105dc648-2ae9-4948-a66b-c79bfc9c573f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:59.142467 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-771709dd-d031-47c2-8bcf-0c5e5f424f06 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:59.143456 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-771709dd-d031-47c2-8bcf-0c5e5f424f06 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:59.153506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-771709dd-d031-47c2-8bcf-0c5e5f424f06 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:59.157572 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-771709dd-d031-47c2-8bcf-0c5e5f424f06 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:59.157572 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-771709dd-d031-47c2-8bcf-0c5e5f424f06 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:59.157572 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-771709dd-d031-47c2-8bcf-0c5e5f424f06 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:59.157572 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-771709dd-d031-47c2-8bcf-0c5e5f424f06 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:59.157904 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-771709dd-d031-47c2-8bcf-0c5e5f424f06 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:59.251416 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-771709dd-d031-47c2-8bcf-0c5e5f424f06 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:59.260673 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-771709dd-d031-47c2-8bcf-0c5e5f424f06 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5" status: 200 len: 1462 microversion: 2.1 time: 0.122442 Sep 04 01:07:59.266971 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-105dc648-2ae9-4948-a66b-c79bfc9c573f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:59.276926 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-105dc648-2ae9-4948-a66b-c79bfc9c573f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1371 microversion: 2.1 time: 0.228642 Sep 04 01:07:59.277784 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 238/492] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:59 2022] GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 => generated 1462 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:07:59.278175 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 255/493] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:59 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1371 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:07:59.412550 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-d7bcbc93-dd6a-4b0f-8aae-9c6e193ee031 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:59.413203 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-d7bcbc93-dd6a-4b0f-8aae-9c6e193ee031 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:59.422746 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d7bcbc93-dd6a-4b0f-8aae-9c6e193ee031 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:59.423257 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d7bcbc93-dd6a-4b0f-8aae-9c6e193ee031 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:59.423747 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d7bcbc93-dd6a-4b0f-8aae-9c6e193ee031 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:59.428626 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d7bcbc93-dd6a-4b0f-8aae-9c6e193ee031 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:59.428626 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d7bcbc93-dd6a-4b0f-8aae-9c6e193ee031 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:59.428626 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d7bcbc93-dd6a-4b0f-8aae-9c6e193ee031 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:59.528820 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-eb417b75-325a-4d82-96b4-83279f4bb739 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:07:59.529779 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-eb417b75-325a-4d82-96b4-83279f4bb739 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:07:59.545039 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-eb417b75-325a-4d82-96b4-83279f4bb739 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:59.545561 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-eb417b75-325a-4d82-96b4-83279f4bb739 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:59.546179 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-eb417b75-325a-4d82-96b4-83279f4bb739 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:59.547055 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-eb417b75-325a-4d82-96b4-83279f4bb739 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:07:59.547659 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-eb417b75-325a-4d82-96b4-83279f4bb739 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:07:59.548460 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-eb417b75-325a-4d82-96b4-83279f4bb739 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:07:59.591009 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-d7bcbc93-dd6a-4b0f-8aae-9c6e193ee031 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:59.604112 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-d7bcbc93-dd6a-4b0f-8aae-9c6e193ee031 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:59.606171 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-d7bcbc93-dd6a-4b0f-8aae-9c6e193ee031 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:59.607493 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-d7bcbc93-dd6a-4b0f-8aae-9c6e193ee031 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1737 microversion: 2.20 time: 0.199798 Sep 04 01:07:59.625114 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 239/494] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:07:59 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1737 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:07:59.712512 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-eb417b75-325a-4d82-96b4-83279f4bb739 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:07:59.720416 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-eb417b75-325a-4d82-96b4-83279f4bb739 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1368 microversion: 2.1 time: 0.195503 Sep 04 01:07:59.721186 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 256/495] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:07:59 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1368 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:08:00.290302 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-8c0adf37-1b81-4989-9feb-6d80674e0691 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:00.290942 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-8c0adf37-1b81-4989-9feb-6d80674e0691 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:00.300019 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-1a0f315e-b279-472e-b86b-4eae6f2d58c9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:00.301890 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-1a0f315e-b279-472e-b86b-4eae6f2d58c9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:00.312835 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1a0f315e-b279-472e-b86b-4eae6f2d58c9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:00.313350 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1a0f315e-b279-472e-b86b-4eae6f2d58c9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:00.314529 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8c0adf37-1b81-4989-9feb-6d80674e0691 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:00.315010 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8c0adf37-1b81-4989-9feb-6d80674e0691 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:00.315605 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1a0f315e-b279-472e-b86b-4eae6f2d58c9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:00.316484 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1a0f315e-b279-472e-b86b-4eae6f2d58c9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:00.317659 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1a0f315e-b279-472e-b86b-4eae6f2d58c9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:00.318479 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8c0adf37-1b81-4989-9feb-6d80674e0691 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:00.319189 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8c0adf37-1b81-4989-9feb-6d80674e0691 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:00.319796 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8c0adf37-1b81-4989-9feb-6d80674e0691 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:00.320423 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1a0f315e-b279-472e-b86b-4eae6f2d58c9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:00.321020 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8c0adf37-1b81-4989-9feb-6d80674e0691 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:00.373706 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8c0adf37-1b81-4989-9feb-6d80674e0691 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:00.374008 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8c0adf37-1b81-4989-9feb-6d80674e0691 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:00.374615 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8c0adf37-1b81-4989-9feb-6d80674e0691 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:00.375218 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.objects.instance [None req-8c0adf37-1b81-4989-9feb-6d80674e0691 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lazy-loading 'fault' on Instance uuid e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 {{(pid=121037) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:08:00.440600 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-8c0adf37-1b81-4989-9feb-6d80674e0691 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:00.450548 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-8c0adf37-1b81-4989-9feb-6d80674e0691 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5" status: 200 len: 1304 microversion: 2.1 time: 0.164057 Sep 04 01:08:00.468851 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 240/496] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:00 2022] GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 => generated 1304 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:00.530854 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-1a0f315e-b279-472e-b86b-4eae6f2d58c9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:00.545642 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-1a0f315e-b279-472e-b86b-4eae6f2d58c9 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1564 microversion: 2.1 time: 0.248927 Sep 04 01:08:00.546589 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 257/497] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:00 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1564 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:00.570858 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-47a78748-c1ef-486f-a0e0-4ef923c96f7f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:00.571401 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-47a78748-c1ef-486f-a0e0-4ef923c96f7f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:00.577542 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-47a78748-c1ef-486f-a0e0-4ef923c96f7f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:00.578030 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-47a78748-c1ef-486f-a0e0-4ef923c96f7f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:00.578464 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-47a78748-c1ef-486f-a0e0-4ef923c96f7f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:00.579411 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-47a78748-c1ef-486f-a0e0-4ef923c96f7f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:00.580070 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-47a78748-c1ef-486f-a0e0-4ef923c96f7f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:00.580801 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-47a78748-c1ef-486f-a0e0-4ef923c96f7f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:00.636338 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-7a196b9f-006d-4b36-a067-fd853fa35662 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:00.636987 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-7a196b9f-006d-4b36-a067-fd853fa35662 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:00.651848 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7a196b9f-006d-4b36-a067-fd853fa35662 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:00.652330 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7a196b9f-006d-4b36-a067-fd853fa35662 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:00.652811 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7a196b9f-006d-4b36-a067-fd853fa35662 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:00.653750 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7a196b9f-006d-4b36-a067-fd853fa35662 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:00.654192 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7a196b9f-006d-4b36-a067-fd853fa35662 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:00.654761 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7a196b9f-006d-4b36-a067-fd853fa35662 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:00.766427 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-47a78748-c1ef-486f-a0e0-4ef923c96f7f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.200422 Sep 04 01:08:00.774619 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 241/498] 10.222.0.42 () {58 vars in 1275 bytes} [Sun Sep 4 01:08:00 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134/os-interface => generated 284 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:08:00.774619 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-2506cacb-d290-497b-9460-f984b3e0cdaa tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:00.774619 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-2506cacb-d290-497b-9460-f984b3e0cdaa tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:00.787724 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2506cacb-d290-497b-9460-f984b3e0cdaa tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:00.791940 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2506cacb-d290-497b-9460-f984b3e0cdaa tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:00.792530 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2506cacb-d290-497b-9460-f984b3e0cdaa tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:00.793428 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2506cacb-d290-497b-9460-f984b3e0cdaa tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:00.803606 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2506cacb-d290-497b-9460-f984b3e0cdaa tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:00.804225 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2506cacb-d290-497b-9460-f984b3e0cdaa tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:00.862978 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-7a196b9f-006d-4b36-a067-fd853fa35662 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:00.874564 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-7a196b9f-006d-4b36-a067-fd853fa35662 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:00.876815 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-7a196b9f-006d-4b36-a067-fd853fa35662 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:00.879025 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-7a196b9f-006d-4b36-a067-fd853fa35662 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1737 microversion: 2.20 time: 0.247053 Sep 04 01:08:00.880278 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 258/499] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:00 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1737 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:08:01.168855 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-2506cacb-d290-497b-9460-f984b3e0cdaa tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:01.177592 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-2506cacb-d290-497b-9460-f984b3e0cdaa tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1368 microversion: 2.1 time: 0.409049 Sep 04 01:08:01.193712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 242/500] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:00 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1368 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:01.482363 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-2c1c948e-0474-4d03-b396-5936ffb7b2f2 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:01.483978 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-2c1c948e-0474-4d03-b396-5936ffb7b2f2 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:01.492058 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2c1c948e-0474-4d03-b396-5936ffb7b2f2 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:01.493354 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2c1c948e-0474-4d03-b396-5936ffb7b2f2 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:01.494118 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2c1c948e-0474-4d03-b396-5936ffb7b2f2 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:01.495118 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2c1c948e-0474-4d03-b396-5936ffb7b2f2 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:01.495748 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2c1c948e-0474-4d03-b396-5936ffb7b2f2 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:01.496391 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2c1c948e-0474-4d03-b396-5936ffb7b2f2 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:01.533879 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2c1c948e-0474-4d03-b396-5936ffb7b2f2 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:01.534982 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2c1c948e-0474-4d03-b396-5936ffb7b2f2 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:01.535944 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2c1c948e-0474-4d03-b396-5936ffb7b2f2 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:01.537211 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.objects.instance [None req-2c1c948e-0474-4d03-b396-5936ffb7b2f2 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lazy-loading 'fault' on Instance uuid e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 {{(pid=121036) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:08:01.659509 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-2c1c948e-0474-4d03-b396-5936ffb7b2f2 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:01.659509 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-2c1c948e-0474-4d03-b396-5936ffb7b2f2 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5" status: 200 len: 1304 microversion: 2.1 time: 0.124970 Sep 04 01:08:01.659509 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 259/501] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:01 2022] GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 => generated 1304 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:01.660336 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-6f231df7-3f8a-4dba-9b20-997610354c6c req-b350f5de-52ff-47a6-b71d-3b12576b08d8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "205e4f5b-43ef-494a-a6ad-64f7445e6b59", "tag": "610a3f9e-176f-46b6-82d0-90b3e3651b7b"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:08:01.660336 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-6f231df7-3f8a-4dba-9b20-997610354c6c req-b350f5de-52ff-47a6-b71d-3b12576b08d8 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:01.660336 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-6f231df7-3f8a-4dba-9b20-997610354c6c req-b350f5de-52ff-47a6-b71d-3b12576b08d8 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:01.660336 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-6f231df7-3f8a-4dba-9b20-997610354c6c req-b350f5de-52ff-47a6-b71d-3b12576b08d8 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:01.662319 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-6f231df7-3f8a-4dba-9b20-997610354c6c req-b350f5de-52ff-47a6-b71d-3b12576b08d8 service nova] Creating event network-changed:610a3f9e-176f-46b6-82d0-90b3e3651b7b for instance 205e4f5b-43ef-494a-a6ad-64f7445e6b59 on hv-cinder-maste Sep 04 01:08:01.662319 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-6f231df7-3f8a-4dba-9b20-997610354c6c req-b350f5de-52ff-47a6-b71d-3b12576b08d8 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031152 Sep 04 01:08:01.662319 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 243/502] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:08:01 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:08:01.907775 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-ddc501af-7e57-48b1-a5e5-0b2ec6b6a32b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:01.908380 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-ddc501af-7e57-48b1-a5e5-0b2ec6b6a32b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:01.914441 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ddc501af-7e57-48b1-a5e5-0b2ec6b6a32b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:01.914921 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ddc501af-7e57-48b1-a5e5-0b2ec6b6a32b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:01.915353 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ddc501af-7e57-48b1-a5e5-0b2ec6b6a32b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:01.916037 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ddc501af-7e57-48b1-a5e5-0b2ec6b6a32b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:01.916486 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ddc501af-7e57-48b1-a5e5-0b2ec6b6a32b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:01.916913 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ddc501af-7e57-48b1-a5e5-0b2ec6b6a32b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:02.073596 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ddc501af-7e57-48b1-a5e5-0b2ec6b6a32b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:02.089817 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ddc501af-7e57-48b1-a5e5-0b2ec6b6a32b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:02.092253 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ddc501af-7e57-48b1-a5e5-0b2ec6b6a32b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:02.093500 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-ddc501af-7e57-48b1-a5e5-0b2ec6b6a32b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1737 microversion: 2.20 time: 0.189608 Sep 04 01:08:02.094060 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 260/503] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:01 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1737 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:08:02.203547 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-df128333-40a8-47de-ab91-0774fa0083b8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:02.204156 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-df128333-40a8-47de-ab91-0774fa0083b8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:02.211734 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-df128333-40a8-47de-ab91-0774fa0083b8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:02.212217 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-df128333-40a8-47de-ab91-0774fa0083b8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:02.212829 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-df128333-40a8-47de-ab91-0774fa0083b8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:02.213716 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-df128333-40a8-47de-ab91-0774fa0083b8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:02.214315 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-df128333-40a8-47de-ab91-0774fa0083b8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:02.214882 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-df128333-40a8-47de-ab91-0774fa0083b8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:02.441519 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-df128333-40a8-47de-ab91-0774fa0083b8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:02.453961 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-df128333-40a8-47de-ab91-0774fa0083b8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1368 microversion: 2.1 time: 0.254694 Sep 04 01:08:02.455185 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 244/504] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:02 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1368 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:02.620969 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-5b033567-5913-45cd-91ea-54858a2b11fe tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:02.621834 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-5b033567-5913-45cd-91ea-54858a2b11fe tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:02.631819 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5b033567-5913-45cd-91ea-54858a2b11fe tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:02.632476 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5b033567-5913-45cd-91ea-54858a2b11fe tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:02.633012 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5b033567-5913-45cd-91ea-54858a2b11fe tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:02.633837 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5b033567-5913-45cd-91ea-54858a2b11fe tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:02.634665 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5b033567-5913-45cd-91ea-54858a2b11fe tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:02.635191 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5b033567-5913-45cd-91ea-54858a2b11fe tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:02.672292 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5b033567-5913-45cd-91ea-54858a2b11fe tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:02.672770 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5b033567-5913-45cd-91ea-54858a2b11fe tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:02.673230 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5b033567-5913-45cd-91ea-54858a2b11fe tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:02.673666 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.objects.instance [None req-5b033567-5913-45cd-91ea-54858a2b11fe tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lazy-loading 'fault' on Instance uuid e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 {{(pid=121036) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:08:02.736039 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-5b033567-5913-45cd-91ea-54858a2b11fe tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:02.747779 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-5b033567-5913-45cd-91ea-54858a2b11fe tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5" status: 200 len: 1304 microversion: 2.1 time: 0.131035 Sep 04 01:08:02.748624 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 261/505] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:02 2022] GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 => generated 1304 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:03.125452 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-f1fcaf57-49aa-4d54-8cac-cee4acef29d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:03.128797 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-f1fcaf57-49aa-4d54-8cac-cee4acef29d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:03.140375 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f1fcaf57-49aa-4d54-8cac-cee4acef29d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:03.145368 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f1fcaf57-49aa-4d54-8cac-cee4acef29d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:03.145368 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f1fcaf57-49aa-4d54-8cac-cee4acef29d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:03.145368 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f1fcaf57-49aa-4d54-8cac-cee4acef29d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:03.145368 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f1fcaf57-49aa-4d54-8cac-cee4acef29d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:03.145757 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f1fcaf57-49aa-4d54-8cac-cee4acef29d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:03.316086 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-f1fcaf57-49aa-4d54-8cac-cee4acef29d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:03.327829 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-f1fcaf57-49aa-4d54-8cac-cee4acef29d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:03.330613 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-f1fcaf57-49aa-4d54-8cac-cee4acef29d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:03.334759 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-f1fcaf57-49aa-4d54-8cac-cee4acef29d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1737 microversion: 2.20 time: 0.213595 Sep 04 01:08:03.334759 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 245/506] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:03 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1737 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:03.478535 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-70d79263-ca14-4b8b-b017-ba96497639c9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:03.479516 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-70d79263-ca14-4b8b-b017-ba96497639c9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:03.486646 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-70d79263-ca14-4b8b-b017-ba96497639c9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:03.487404 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-70d79263-ca14-4b8b-b017-ba96497639c9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:03.488052 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-70d79263-ca14-4b8b-b017-ba96497639c9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:03.489026 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-70d79263-ca14-4b8b-b017-ba96497639c9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:03.489722 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-70d79263-ca14-4b8b-b017-ba96497639c9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:03.490376 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-70d79263-ca14-4b8b-b017-ba96497639c9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:03.661332 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-5658ff33-574f-428b-9f3c-1b0caab405de req-0f760a72-fe93-4c15-8c5c-078c4bf2aebb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2570ee6c-f335-41bc-8e67-f89a62b28134", "tag": "62f5079a-1d7d-4c5c-bdcd-0af337b6235b"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:08:03.673105 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-5658ff33-574f-428b-9f3c-1b0caab405de req-0f760a72-fe93-4c15-8c5c-078c4bf2aebb service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:03.675085 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-70d79263-ca14-4b8b-b017-ba96497639c9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:03.680868 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-5658ff33-574f-428b-9f3c-1b0caab405de req-0f760a72-fe93-4c15-8c5c-078c4bf2aebb service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:03.681352 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-5658ff33-574f-428b-9f3c-1b0caab405de req-0f760a72-fe93-4c15-8c5c-078c4bf2aebb service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:03.686146 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-70d79263-ca14-4b8b-b017-ba96497639c9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1368 microversion: 2.1 time: 0.211978 Sep 04 01:08:03.701170 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-5658ff33-574f-428b-9f3c-1b0caab405de req-0f760a72-fe93-4c15-8c5c-078c4bf2aebb service nova] Creating event network-changed:62f5079a-1d7d-4c5c-bdcd-0af337b6235b for instance 2570ee6c-f335-41bc-8e67-f89a62b28134 on hv-cinder-maste Sep 04 01:08:03.702729 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 262/507] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:03 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1368 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:03.710351 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-5658ff33-574f-428b-9f3c-1b0caab405de req-0f760a72-fe93-4c15-8c5c-078c4bf2aebb service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.052956 Sep 04 01:08:03.711423 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 246/508] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:08:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:08:03.773361 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-5ca3ba1d-bd83-426c-917e-0835da2b8993 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:03.774278 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-5ca3ba1d-bd83-426c-917e-0835da2b8993 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:03.788305 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5ca3ba1d-bd83-426c-917e-0835da2b8993 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:03.789205 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5ca3ba1d-bd83-426c-917e-0835da2b8993 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:03.790091 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5ca3ba1d-bd83-426c-917e-0835da2b8993 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:03.790815 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5ca3ba1d-bd83-426c-917e-0835da2b8993 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:03.791742 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5ca3ba1d-bd83-426c-917e-0835da2b8993 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:03.792182 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5ca3ba1d-bd83-426c-917e-0835da2b8993 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:03.853974 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5ca3ba1d-bd83-426c-917e-0835da2b8993 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:03.854369 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5ca3ba1d-bd83-426c-917e-0835da2b8993 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:03.854990 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5ca3ba1d-bd83-426c-917e-0835da2b8993 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:03.855447 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.objects.instance [None req-5ca3ba1d-bd83-426c-917e-0835da2b8993 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lazy-loading 'fault' on Instance uuid e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 {{(pid=121036) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:08:03.908325 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-5ca3ba1d-bd83-426c-917e-0835da2b8993 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:03.917202 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-5ca3ba1d-bd83-426c-917e-0835da2b8993 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5" status: 200 len: 1304 microversion: 2.1 time: 0.149236 Sep 04 01:08:03.918070 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 263/509] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:03 2022] GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 => generated 1304 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:04.360207 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-8e57944a-3202-4ea1-83bc-c6d581ec18ed tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:04.361148 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-8e57944a-3202-4ea1-83bc-c6d581ec18ed tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:04.369494 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8e57944a-3202-4ea1-83bc-c6d581ec18ed tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:04.370095 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8e57944a-3202-4ea1-83bc-c6d581ec18ed tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:04.370602 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8e57944a-3202-4ea1-83bc-c6d581ec18ed tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:04.371426 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8e57944a-3202-4ea1-83bc-c6d581ec18ed tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:04.371945 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8e57944a-3202-4ea1-83bc-c6d581ec18ed tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:04.372649 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8e57944a-3202-4ea1-83bc-c6d581ec18ed tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:04.528291 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-8e57944a-3202-4ea1-83bc-c6d581ec18ed tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:04.539379 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-8e57944a-3202-4ea1-83bc-c6d581ec18ed tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:04.541400 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-8e57944a-3202-4ea1-83bc-c6d581ec18ed tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:04.542823 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-8e57944a-3202-4ea1-83bc-c6d581ec18ed tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1737 microversion: 2.20 time: 0.187115 Sep 04 01:08:04.543563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 247/510] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:04 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1737 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:08:04.727603 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-101c1f2b-26cb-40ed-a251-36db77a74efe tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:04.728865 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-101c1f2b-26cb-40ed-a251-36db77a74efe tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:04.737429 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-101c1f2b-26cb-40ed-a251-36db77a74efe tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:04.738128 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-101c1f2b-26cb-40ed-a251-36db77a74efe tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:04.739471 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-101c1f2b-26cb-40ed-a251-36db77a74efe tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:04.740994 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-101c1f2b-26cb-40ed-a251-36db77a74efe tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:04.741647 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-101c1f2b-26cb-40ed-a251-36db77a74efe tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:04.742368 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-101c1f2b-26cb-40ed-a251-36db77a74efe tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:04.948426 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-111060f7-9e9a-46bb-ad9f-de88b75c8f0a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:04.949168 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-111060f7-9e9a-46bb-ad9f-de88b75c8f0a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:04.960467 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-111060f7-9e9a-46bb-ad9f-de88b75c8f0a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:04.961103 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-111060f7-9e9a-46bb-ad9f-de88b75c8f0a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:04.961674 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-111060f7-9e9a-46bb-ad9f-de88b75c8f0a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:04.963278 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-111060f7-9e9a-46bb-ad9f-de88b75c8f0a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:04.963278 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-111060f7-9e9a-46bb-ad9f-de88b75c8f0a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:04.963478 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-111060f7-9e9a-46bb-ad9f-de88b75c8f0a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:05.005798 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-111060f7-9e9a-46bb-ad9f-de88b75c8f0a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:05.006818 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-111060f7-9e9a-46bb-ad9f-de88b75c8f0a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:05.007641 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-111060f7-9e9a-46bb-ad9f-de88b75c8f0a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:05.008284 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.objects.instance [None req-111060f7-9e9a-46bb-ad9f-de88b75c8f0a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lazy-loading 'fault' on Instance uuid e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 {{(pid=121037) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:08:05.040667 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-101c1f2b-26cb-40ed-a251-36db77a74efe tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:05.049798 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-101c1f2b-26cb-40ed-a251-36db77a74efe tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1368 microversion: 2.1 time: 0.326829 Sep 04 01:08:05.061121 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 264/511] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:04 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1368 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:05.076636 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-111060f7-9e9a-46bb-ad9f-de88b75c8f0a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:05.084565 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-111060f7-9e9a-46bb-ad9f-de88b75c8f0a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5" status: 200 len: 1304 microversion: 2.1 time: 0.140020 Sep 04 01:08:05.085158 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 248/512] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:04 2022] GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 => generated 1304 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:08:05.573561 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-fc099bcb-a9fe-46e1-82ff-4761922b58a8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:05.574173 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-fc099bcb-a9fe-46e1-82ff-4761922b58a8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:05.580519 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fc099bcb-a9fe-46e1-82ff-4761922b58a8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:05.581004 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fc099bcb-a9fe-46e1-82ff-4761922b58a8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:05.584806 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fc099bcb-a9fe-46e1-82ff-4761922b58a8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:05.584806 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fc099bcb-a9fe-46e1-82ff-4761922b58a8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:05.584806 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fc099bcb-a9fe-46e1-82ff-4761922b58a8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:05.584806 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fc099bcb-a9fe-46e1-82ff-4761922b58a8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:05.786480 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-fc099bcb-a9fe-46e1-82ff-4761922b58a8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:05.798035 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-fc099bcb-a9fe-46e1-82ff-4761922b58a8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:05.805295 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-fc099bcb-a9fe-46e1-82ff-4761922b58a8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:05.806515 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-fc099bcb-a9fe-46e1-82ff-4761922b58a8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1737 microversion: 2.20 time: 0.237128 Sep 04 01:08:05.807256 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 265/513] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:05 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1737 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:06.074258 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-7eb97ac9-d9f1-48f8-8ca1-5a4dbba220b1 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:06.075849 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-7eb97ac9-d9f1-48f8-8ca1-5a4dbba220b1 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:06.090464 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7eb97ac9-d9f1-48f8-8ca1-5a4dbba220b1 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:06.090868 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7eb97ac9-d9f1-48f8-8ca1-5a4dbba220b1 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:06.091317 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7eb97ac9-d9f1-48f8-8ca1-5a4dbba220b1 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:06.092404 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7eb97ac9-d9f1-48f8-8ca1-5a4dbba220b1 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:06.092859 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7eb97ac9-d9f1-48f8-8ca1-5a4dbba220b1 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:06.093320 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7eb97ac9-d9f1-48f8-8ca1-5a4dbba220b1 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:06.109969 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-053a4652-6db8-4b0d-a54f-8b4412a49501 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:06.110867 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-053a4652-6db8-4b0d-a54f-8b4412a49501 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:06.126623 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-053a4652-6db8-4b0d-a54f-8b4412a49501 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:06.133106 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-053a4652-6db8-4b0d-a54f-8b4412a49501 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:06.133106 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-053a4652-6db8-4b0d-a54f-8b4412a49501 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:06.133106 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-053a4652-6db8-4b0d-a54f-8b4412a49501 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:06.133106 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-053a4652-6db8-4b0d-a54f-8b4412a49501 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:06.133682 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-053a4652-6db8-4b0d-a54f-8b4412a49501 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:06.157897 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.wsgi [None req-053a4652-6db8-4b0d-a54f-8b4412a49501 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] HTTP exception thrown: Instance e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 could not be found. Sep 04 01:08:06.160716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-053a4652-6db8-4b0d-a54f-8b4412a49501 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Request method failure captured: Sep 04 01:08:06.160716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: request: GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 HTTP/1.1 Sep 04 01:08:06.160716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept: application/json Sep 04 01:08:06.160716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept-Encoding: identity Sep 04 01:08:06.160716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Connection: close Sep 04 01:08:06.160716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Length: 0 Sep 04 01:08:06.160716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Type: application/json Sep 04 01:08:06.160716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Host: 10.222.0.42 Sep 04 01:08:06.160716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Openstack-System-Scope: None Sep 04 01:08:06.160716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: User-Agent: python-urllib3/1.26.12 Sep 04 01:08:06.160716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Auth-Token: gAAAAABjE_pNrD102VA_M1LsqvXxEL4ASLRJkOZs66JwIfhr4LA66jVzzTUo4pav2dTmV9yYU0q13x5ZoPzE1w7gETzsYCVAdHqPe0UHBSCrxKJJDEP1hNwvK2NOJp8ou8Lm331yT-EoM27cMHSV797nDbWpQ9djpIka044FZeyn_90GUP8cR7o Sep 04 01:08:06.160716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Id: None Sep 04 01:08:06.160716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Name: None Sep 04 01:08:06.160716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Identity-Status: Confirmed Sep 04 01:08:06.160716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Is-Admin-Project: True Sep 04 01:08:06.160716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Id: default Sep 04 01:08:06.160716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Name: Default Sep 04 01:08:06.160716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Id: bcf6c738f8d842f99d2eec3b896c5b47 Sep 04 01:08:06.160716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Name: tempest-VolumesActionsTest-1122363192 Sep 04 01:08:06.160716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Role: reader,member Sep 04 01:08:06.160716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Roles: reader,member Sep 04 01:08:06.162667 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/bcf6c738f8d842f99d2eec3b896c5b47"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_bcf6c738f8d842f99d2eec3b896c5b47", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/bcf6c738f8d842f99d2eec3b896c5b47"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/bcf6c738f8d842f99d2eec3b896c5b47"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:08:06.162667 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant: tempest-VolumesActionsTest-1122363192 Sep 04 01:08:06.162667 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Id: bcf6c738f8d842f99d2eec3b896c5b47 Sep 04 01:08:06.162667 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Name: tempest-VolumesActionsTest-1122363192 Sep 04 01:08:06.162667 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User: tempest-VolumesActionsTest-1122363192-project Sep 04 01:08:06.162667 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Id: default Sep 04 01:08:06.162667 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Name: Default Sep 04 01:08:06.162667 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Id: a6cf41aaddce4a86825bff00477793f9 Sep 04 01:08:06.162667 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Name: tempest-VolumesActionsTest-1122363192-project Sep 04 01:08:06.162667 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: method: > Sep 04 01:08:06.162667 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: exception: Instance e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 could not be found. Sep 04 01:08:06.162667 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: action_args: {'id': 'e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5'} Sep 04 01:08:06.163610 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:08:06.163610 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:08:06.163610 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:08:06.163610 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:08:06.163610 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:08:06.163610 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:08:06.163610 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:08:06.163610 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:08:06.163610 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:08:06.163610 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:08:06.163610 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 could not be found. Sep 04 01:08:06.163610 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:08:06.163610 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:08:06.163610 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:08:06.163610 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:08:06.163610 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:08:06.163610 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:08:06.163610 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:08:06.163610 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:08:06.163610 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:08:06.165727 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:08:06.165727 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 04 01:08:06.165727 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 04 01:08:06.165727 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:08:06.165727 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:08:06.165727 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:08:06.165727 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:08:06.165727 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 could not be found. Sep 04 01:08:06.165727 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:08:06.165727 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:08:06.165727 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:08:06.165727 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:08:06.165727 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:08:06.165727 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:08:06.165727 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:08:06.165727 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:08:06.165727 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 could not be found. Sep 04 01:08:06.165727 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:08:06.167964 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-053a4652-6db8-4b0d-a54f-8b4412a49501 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Returning 404 to user: Instance e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 could not be found. {{(pid=121036) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:08:06.168989 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-053a4652-6db8-4b0d-a54f-8b4412a49501 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5" status: 404 len: 111 microversion: 2.1 time: 0.063777 Sep 04 01:08:06.170069 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 266/514] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:06 2022] GET /compute/v2.1/servers/e1a5a54b-ef31-408d-ac1c-0a78efd9a2b5 => generated 111 bytes in 65 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 04 01:08:06.252128 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-35b3b1b3-3e8c-4e36-896b-a3c9eb81c961 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1140730620", "imageRef": "a11315c3-ce81-478b-91e4-5353f189dfa7", "flavorRef": "452", "networks": [{"uuid": "7e04f73d-bad8-4860-94c2-08b2d4e17fa9"}]}} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:08:06.340538 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.network.neutron [None req-35b3b1b3-3e8c-4e36-896b-a3c9eb81c961 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] validate_networks() for [('7e04f73d-bad8-4860-94c2-08b2d4e17fa9', None, None, None, None, None)] {{(pid=121036) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 04 01:08:06.364694 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-7eb97ac9-d9f1-48f8-8ca1-5a4dbba220b1 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:06.374268 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-7eb97ac9-d9f1-48f8-8ca1-5a4dbba220b1 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1368 microversion: 2.1 time: 0.304346 Sep 04 01:08:06.383603 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 249/515] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:06 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1368 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:06.804719 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-35b3b1b3-3e8c-4e36-896b-a3c9eb81c961 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Flavor limits 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 04 01:08:06.805229 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-35b3b1b3-3e8c-4e36-896b-a3c9eb81c961 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Image limits 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 04 01:08:06.806308 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-35b3b1b3-3e8c-4e36-896b-a3c9eb81c961 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Flavor pref 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 04 01:08:06.806308 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-35b3b1b3-3e8c-4e36-896b-a3c9eb81c961 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Image pref 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 04 01:08:06.806514 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-35b3b1b3-3e8c-4e36-896b-a3c9eb81c961 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 04 01:08:06.815301 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-35b3b1b3-3e8c-4e36-896b-a3c9eb81c961 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:06.815849 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-35b3b1b3-3e8c-4e36-896b-a3c9eb81c961 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:06.816286 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-35b3b1b3-3e8c-4e36-896b-a3c9eb81c961 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:06.817026 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-35b3b1b3-3e8c-4e36-896b-a3c9eb81c961 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:06.817582 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-35b3b1b3-3e8c-4e36-896b-a3c9eb81c961 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:06.818001 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-35b3b1b3-3e8c-4e36-896b-a3c9eb81c961 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:06.829430 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:06.833026 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-9373a16f-afe6-4f23-aa53-2d0e05a9b653 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:06.833655 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-9373a16f-afe6-4f23-aa53-2d0e05a9b653 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:06.836698 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-35b3b1b3-3e8c-4e36-896b-a3c9eb81c961 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Getting quotas for project bcf6c738f8d842f99d2eec3b896c5b47. Resources: {'ram', 'instances', 'cores'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 04 01:08:06.840106 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9373a16f-afe6-4f23-aa53-2d0e05a9b653 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:06.843892 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9373a16f-afe6-4f23-aa53-2d0e05a9b653 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:06.844013 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-35b3b1b3-3e8c-4e36-896b-a3c9eb81c961 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Getting quotas for user a6cf41aaddce4a86825bff00477793f9 and project bcf6c738f8d842f99d2eec3b896c5b47. Resources: {'ram', 'instances', 'cores'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 04 01:08:06.844106 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9373a16f-afe6-4f23-aa53-2d0e05a9b653 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:06.844106 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9373a16f-afe6-4f23-aa53-2d0e05a9b653 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:06.844106 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9373a16f-afe6-4f23-aa53-2d0e05a9b653 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:06.844106 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9373a16f-afe6-4f23-aa53-2d0e05a9b653 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:06.853096 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-35b3b1b3-3e8c-4e36-896b-a3c9eb81c961 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Going to run 1 instances... {{(pid=121036) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 04 01:08:06.855766 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-35b3b1b3-3e8c-4e36-896b-a3c9eb81c961 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a11315c3-ce81-478b-91e4-5353f189dfa7',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121036) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 04 01:08:06.971603 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-35b3b1b3-3e8c-4e36-896b-a3c9eb81c961 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.723505 Sep 04 01:08:06.972454 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 267/516] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 4 01:08:06 2022] POST /compute/v2.1/servers => generated 380 bytes in 725 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 04 01:08:06.987231 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-9373a16f-afe6-4f23-aa53-2d0e05a9b653 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:06.987744 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-fea17228-187d-4df2-86e3-cf2759b9f204 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:06.988373 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-fea17228-187d-4df2-86e3-cf2759b9f204 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:07.008497 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-9373a16f-afe6-4f23-aa53-2d0e05a9b653 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:07.010187 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-9373a16f-afe6-4f23-aa53-2d0e05a9b653 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:07.012642 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-9373a16f-afe6-4f23-aa53-2d0e05a9b653 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1737 microversion: 2.20 time: 0.182410 Sep 04 01:08:07.012642 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 250/517] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:06 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1737 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:08:07.046105 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-fea17228-187d-4df2-86e3-cf2759b9f204 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:07.053356 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-fea17228-187d-4df2-86e3-cf2759b9f204 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 200 len: 1214 microversion: 2.1 time: 0.071994 Sep 04 01:08:07.053940 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 268/518] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:06 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 1214 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:07.407732 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-598a5dba-45c7-4fd9-b872-435329e35d81 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:07.408650 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-598a5dba-45c7-4fd9-b872-435329e35d81 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:07.415607 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-598a5dba-45c7-4fd9-b872-435329e35d81 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:07.420208 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-598a5dba-45c7-4fd9-b872-435329e35d81 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:07.420208 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-598a5dba-45c7-4fd9-b872-435329e35d81 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:07.420208 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-598a5dba-45c7-4fd9-b872-435329e35d81 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:07.420208 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-598a5dba-45c7-4fd9-b872-435329e35d81 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:07.420519 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-598a5dba-45c7-4fd9-b872-435329e35d81 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:07.564303 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-598a5dba-45c7-4fd9-b872-435329e35d81 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:07.568830 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=100 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:07.573273 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-598a5dba-45c7-4fd9-b872-435329e35d81 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1368 microversion: 2.1 time: 0.169637 Sep 04 01:08:07.573839 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 251/519] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:07 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1368 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:08.036376 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-bde0ad3a-bb00-444b-9586-3ce82c3054db tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:08.036971 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-bde0ad3a-bb00-444b-9586-3ce82c3054db tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:08.043703 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bde0ad3a-bb00-444b-9586-3ce82c3054db tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:08.044176 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bde0ad3a-bb00-444b-9586-3ce82c3054db tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:08.044674 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bde0ad3a-bb00-444b-9586-3ce82c3054db tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:08.045690 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bde0ad3a-bb00-444b-9586-3ce82c3054db tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:08.045889 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bde0ad3a-bb00-444b-9586-3ce82c3054db tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:08.046326 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bde0ad3a-bb00-444b-9586-3ce82c3054db tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:08.076122 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-21633ccc-ffb3-44c0-b96f-57e2aea5f4c1 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:08.076658 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-21633ccc-ffb3-44c0-b96f-57e2aea5f4c1 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:08.083095 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-21633ccc-ffb3-44c0-b96f-57e2aea5f4c1 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:08.083569 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-21633ccc-ffb3-44c0-b96f-57e2aea5f4c1 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:08.083998 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-21633ccc-ffb3-44c0-b96f-57e2aea5f4c1 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:08.084687 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-21633ccc-ffb3-44c0-b96f-57e2aea5f4c1 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:08.085160 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-21633ccc-ffb3-44c0-b96f-57e2aea5f4c1 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:08.085583 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-21633ccc-ffb3-44c0-b96f-57e2aea5f4c1 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:08.153648 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-21633ccc-ffb3-44c0-b96f-57e2aea5f4c1 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:08.163710 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-21633ccc-ffb3-44c0-b96f-57e2aea5f4c1 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 200 len: 1218 microversion: 2.1 time: 0.091750 Sep 04 01:08:08.164285 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 252/520] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:08 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 1218 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:08:08.190991 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-bde0ad3a-bb00-444b-9586-3ce82c3054db tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:08.199393 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-bde0ad3a-bb00-444b-9586-3ce82c3054db tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:08.200836 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-bde0ad3a-bb00-444b-9586-3ce82c3054db tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:08.202025 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-bde0ad3a-bb00-444b-9586-3ce82c3054db tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1737 microversion: 2.20 time: 0.169682 Sep 04 01:08:08.202576 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 269/521] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:08 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1737 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:08.596900 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-38d0432c-ce22-405e-a76e-ac9535fa09c4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:08.597467 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-38d0432c-ce22-405e-a76e-ac9535fa09c4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:08.603986 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-38d0432c-ce22-405e-a76e-ac9535fa09c4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:08.604468 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-38d0432c-ce22-405e-a76e-ac9535fa09c4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:08.604912 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-38d0432c-ce22-405e-a76e-ac9535fa09c4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:08.605638 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-38d0432c-ce22-405e-a76e-ac9535fa09c4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:08.606089 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-38d0432c-ce22-405e-a76e-ac9535fa09c4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:08.606516 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-38d0432c-ce22-405e-a76e-ac9535fa09c4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:08.736607 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-38d0432c-ce22-405e-a76e-ac9535fa09c4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:08.744712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-38d0432c-ce22-405e-a76e-ac9535fa09c4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1368 microversion: 2.1 time: 0.151749 Sep 04 01:08:08.745299 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 253/522] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:08 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1368 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:09.187299 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-f22bd671-be10-4944-95c7-ed5598eb00a8 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:09.187909 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-f22bd671-be10-4944-95c7-ed5598eb00a8 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:09.194517 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f22bd671-be10-4944-95c7-ed5598eb00a8 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:09.194996 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f22bd671-be10-4944-95c7-ed5598eb00a8 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:09.195434 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f22bd671-be10-4944-95c7-ed5598eb00a8 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:09.196117 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f22bd671-be10-4944-95c7-ed5598eb00a8 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:09.196567 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f22bd671-be10-4944-95c7-ed5598eb00a8 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:09.196993 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f22bd671-be10-4944-95c7-ed5598eb00a8 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:09.228329 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-ae5cec14-9228-4096-94a2-3d62275912ff tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:09.229174 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-ae5cec14-9228-4096-94a2-3d62275912ff tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:09.236490 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ae5cec14-9228-4096-94a2-3d62275912ff tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:09.237143 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ae5cec14-9228-4096-94a2-3d62275912ff tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:09.237863 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ae5cec14-9228-4096-94a2-3d62275912ff tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:09.238933 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ae5cec14-9228-4096-94a2-3d62275912ff tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:09.239782 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ae5cec14-9228-4096-94a2-3d62275912ff tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:09.240435 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ae5cec14-9228-4096-94a2-3d62275912ff tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:09.268155 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-f22bd671-be10-4944-95c7-ed5598eb00a8 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:09.276648 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-f22bd671-be10-4944-95c7-ed5598eb00a8 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 200 len: 1210 microversion: 2.1 time: 0.093225 Sep 04 01:08:09.277241 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 270/523] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:09 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 1210 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:09.454156 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-ae5cec14-9228-4096-94a2-3d62275912ff tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:09.463270 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-ae5cec14-9228-4096-94a2-3d62275912ff tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:09.466647 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-ae5cec14-9228-4096-94a2-3d62275912ff tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:09.467824 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-ae5cec14-9228-4096-94a2-3d62275912ff tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1719 microversion: 2.20 time: 0.243631 Sep 04 01:08:09.469002 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 254/524] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:09 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1719 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:09.536926 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-248185e3-1b9e-4919-bd67-c541f973fc49 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:09.537491 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-248185e3-1b9e-4919-bd67-c541f973fc49 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: 2808d334-c39c-4a09-88c7-c1711cce4215] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:09.547051 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-248185e3-1b9e-4919-bd67-c541f973fc49 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:09.547537 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-248185e3-1b9e-4919-bd67-c541f973fc49 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:09.548128 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-248185e3-1b9e-4919-bd67-c541f973fc49 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:09.548879 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-248185e3-1b9e-4919-bd67-c541f973fc49 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:09.549361 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-248185e3-1b9e-4919-bd67-c541f973fc49 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:09.549783 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-248185e3-1b9e-4919-bd67-c541f973fc49 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:09.683208 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.objects.instance [None req-248185e3-1b9e-4919-bd67-c541f973fc49 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lazy-loading 'services' on Instance uuid 2808d334-c39c-4a09-88c7-c1711cce4215 {{(pid=121036) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:08:09.767742 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-ee4f2d1c-f2bb-4db8-b40f-60d47353db7a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:09.768360 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-ee4f2d1c-f2bb-4db8-b40f-60d47353db7a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:09.775216 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ee4f2d1c-f2bb-4db8-b40f-60d47353db7a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:09.775698 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ee4f2d1c-f2bb-4db8-b40f-60d47353db7a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:09.776132 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ee4f2d1c-f2bb-4db8-b40f-60d47353db7a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:09.776820 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ee4f2d1c-f2bb-4db8-b40f-60d47353db7a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:09.777343 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ee4f2d1c-f2bb-4db8-b40f-60d47353db7a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:09.777809 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ee4f2d1c-f2bb-4db8-b40f-60d47353db7a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:09.876885 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:09.913066 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-ee4f2d1c-f2bb-4db8-b40f-60d47353db7a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:09.922419 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-ee4f2d1c-f2bb-4db8-b40f-60d47353db7a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1368 microversion: 2.1 time: 0.158508 Sep 04 01:08:09.922977 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 255/525] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:09 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1368 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:08:09.969416 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-248185e3-1b9e-4919-bd67-c541f973fc49 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "DELETE /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215/os-volume_attachments/19c48c1c-9d94-45af-a46a-d4af0c2d845c" status: 202 len: 0 microversion: 2.1 time: 0.437711 Sep 04 01:08:09.970229 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 271/526] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Sep 4 01:08:09 2022] DELETE /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215/os-volume_attachments/19c48c1c-9d94-45af-a46a-d4af0c2d845c => generated 0 bytes in 439 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 04 01:08:10.300463 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-bcbff5fc-6a06-4954-84ff-875a27b74789 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:10.301127 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-bcbff5fc-6a06-4954-84ff-875a27b74789 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:10.306995 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-bcbff5fc-6a06-4954-84ff-875a27b74789 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:10.307469 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-bcbff5fc-6a06-4954-84ff-875a27b74789 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:10.307896 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-bcbff5fc-6a06-4954-84ff-875a27b74789 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:10.308574 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-bcbff5fc-6a06-4954-84ff-875a27b74789 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:10.309139 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-bcbff5fc-6a06-4954-84ff-875a27b74789 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:10.309560 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-bcbff5fc-6a06-4954-84ff-875a27b74789 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:10.369833 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-bcbff5fc-6a06-4954-84ff-875a27b74789 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:10.377318 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-bcbff5fc-6a06-4954-84ff-875a27b74789 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 200 len: 1266 microversion: 2.1 time: 0.080929 Sep 04 01:08:10.377938 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 256/527] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:10 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 1266 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:10.495978 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-6e639eb9-a214-42d4-a206-1c9e5e2188be tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:10.496666 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-6e639eb9-a214-42d4-a206-1c9e5e2188be tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:10.503781 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6e639eb9-a214-42d4-a206-1c9e5e2188be tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:10.504780 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6e639eb9-a214-42d4-a206-1c9e5e2188be tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:10.505415 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6e639eb9-a214-42d4-a206-1c9e5e2188be tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:10.506101 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6e639eb9-a214-42d4-a206-1c9e5e2188be tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:10.506549 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6e639eb9-a214-42d4-a206-1c9e5e2188be tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:10.506973 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6e639eb9-a214-42d4-a206-1c9e5e2188be tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:10.643144 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-6e639eb9-a214-42d4-a206-1c9e5e2188be tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:10.652818 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-6e639eb9-a214-42d4-a206-1c9e5e2188be tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:10.654383 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-6e639eb9-a214-42d4-a206-1c9e5e2188be tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:10.655549 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-6e639eb9-a214-42d4-a206-1c9e5e2188be tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1719 microversion: 2.20 time: 0.164206 Sep 04 01:08:10.656109 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 272/528] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:10 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1719 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:10.945952 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-7b2c7bc9-7544-42ce-87de-21eb6f3ff51a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:10.946563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-7b2c7bc9-7544-42ce-87de-21eb6f3ff51a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:10.952639 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7b2c7bc9-7544-42ce-87de-21eb6f3ff51a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:10.953169 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7b2c7bc9-7544-42ce-87de-21eb6f3ff51a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:10.953601 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7b2c7bc9-7544-42ce-87de-21eb6f3ff51a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:10.954294 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7b2c7bc9-7544-42ce-87de-21eb6f3ff51a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:10.954741 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7b2c7bc9-7544-42ce-87de-21eb6f3ff51a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:10.955161 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7b2c7bc9-7544-42ce-87de-21eb6f3ff51a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:11.094729 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-7b2c7bc9-7544-42ce-87de-21eb6f3ff51a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:11.103305 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-7b2c7bc9-7544-42ce-87de-21eb6f3ff51a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1368 microversion: 2.1 time: 0.161775 Sep 04 01:08:11.106617 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 257/529] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:10 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1368 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:11.400640 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-dfd7610f-29a2-4e97-98d9-cfd54a116b23 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:11.401190 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-dfd7610f-29a2-4e97-98d9-cfd54a116b23 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:11.407276 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dfd7610f-29a2-4e97-98d9-cfd54a116b23 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:11.407753 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dfd7610f-29a2-4e97-98d9-cfd54a116b23 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:11.408189 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dfd7610f-29a2-4e97-98d9-cfd54a116b23 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:11.408875 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dfd7610f-29a2-4e97-98d9-cfd54a116b23 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:11.409344 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dfd7610f-29a2-4e97-98d9-cfd54a116b23 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:11.409767 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dfd7610f-29a2-4e97-98d9-cfd54a116b23 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:11.476744 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-dfd7610f-29a2-4e97-98d9-cfd54a116b23 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:11.484685 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-dfd7610f-29a2-4e97-98d9-cfd54a116b23 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 200 len: 1266 microversion: 2.1 time: 0.087735 Sep 04 01:08:11.485369 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 273/530] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:11 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 1266 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:11.682991 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-9ec5c1a9-ff01-4a45-9b3a-0a1f7f3ada18 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:11.683514 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-9ec5c1a9-ff01-4a45-9b3a-0a1f7f3ada18 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:11.692412 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9ec5c1a9-ff01-4a45-9b3a-0a1f7f3ada18 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:11.693515 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9ec5c1a9-ff01-4a45-9b3a-0a1f7f3ada18 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:11.694113 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9ec5c1a9-ff01-4a45-9b3a-0a1f7f3ada18 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:11.695038 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9ec5c1a9-ff01-4a45-9b3a-0a1f7f3ada18 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:11.695499 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9ec5c1a9-ff01-4a45-9b3a-0a1f7f3ada18 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:11.695928 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9ec5c1a9-ff01-4a45-9b3a-0a1f7f3ada18 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:11.870238 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-9ec5c1a9-ff01-4a45-9b3a-0a1f7f3ada18 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:11.878786 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-9ec5c1a9-ff01-4a45-9b3a-0a1f7f3ada18 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:11.880391 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-9ec5c1a9-ff01-4a45-9b3a-0a1f7f3ada18 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:11.881594 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-9ec5c1a9-ff01-4a45-9b3a-0a1f7f3ada18 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1719 microversion: 2.20 time: 0.202867 Sep 04 01:08:11.882241 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 258/531] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:11 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1719 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:12.132249 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-4b32849b-b70f-42c8-9b44-88a299c3bf10 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:12.133366 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-4b32849b-b70f-42c8-9b44-88a299c3bf10 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:12.144724 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4b32849b-b70f-42c8-9b44-88a299c3bf10 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:12.145926 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4b32849b-b70f-42c8-9b44-88a299c3bf10 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:12.146863 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4b32849b-b70f-42c8-9b44-88a299c3bf10 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:12.148688 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4b32849b-b70f-42c8-9b44-88a299c3bf10 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:12.149488 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4b32849b-b70f-42c8-9b44-88a299c3bf10 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:12.150503 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4b32849b-b70f-42c8-9b44-88a299c3bf10 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:12.313743 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-4b32849b-b70f-42c8-9b44-88a299c3bf10 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:12.321608 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-4b32849b-b70f-42c8-9b44-88a299c3bf10 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1368 microversion: 2.1 time: 0.196683 Sep 04 01:08:12.322166 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 274/532] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:12 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1368 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:12.511276 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-0b96edf3-f070-4163-b592-17a926e50cc8 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:12.512022 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-0b96edf3-f070-4163-b592-17a926e50cc8 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:12.522487 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0b96edf3-f070-4163-b592-17a926e50cc8 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:12.523650 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0b96edf3-f070-4163-b592-17a926e50cc8 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:12.524286 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0b96edf3-f070-4163-b592-17a926e50cc8 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:12.526323 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0b96edf3-f070-4163-b592-17a926e50cc8 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:12.527162 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0b96edf3-f070-4163-b592-17a926e50cc8 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:12.527677 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0b96edf3-f070-4163-b592-17a926e50cc8 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:12.652614 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-0b96edf3-f070-4163-b592-17a926e50cc8 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:12.660689 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-0b96edf3-f070-4163-b592-17a926e50cc8 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 200 len: 1266 microversion: 2.1 time: 0.156598 Sep 04 01:08:12.661252 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 259/533] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:12 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 1266 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:12.846361 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-623a293a-86ab-4633-a2db-10aad7634b00 req-9fd722e3-f135-42b4-8ce5-466abaedb7e4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "205e4f5b-43ef-494a-a6ad-64f7445e6b59", "name": "network-vif-plugged", "status": "completed", "tag": "610a3f9e-176f-46b6-82d0-90b3e3651b7b"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:08:12.859939 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-623a293a-86ab-4633-a2db-10aad7634b00 req-9fd722e3-f135-42b4-8ce5-466abaedb7e4 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:12.860427 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-623a293a-86ab-4633-a2db-10aad7634b00 req-9fd722e3-f135-42b4-8ce5-466abaedb7e4 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:12.860860 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-623a293a-86ab-4633-a2db-10aad7634b00 req-9fd722e3-f135-42b4-8ce5-466abaedb7e4 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:12.875417 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-623a293a-86ab-4633-a2db-10aad7634b00 req-9fd722e3-f135-42b4-8ce5-466abaedb7e4 service nova] Creating event network-vif-plugged:610a3f9e-176f-46b6-82d0-90b3e3651b7b for instance 205e4f5b-43ef-494a-a6ad-64f7445e6b59 on hv-cinder-maste Sep 04 01:08:12.882934 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-623a293a-86ab-4633-a2db-10aad7634b00 req-9fd722e3-f135-42b4-8ce5-466abaedb7e4 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042824 Sep 04 01:08:12.883686 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 275/534] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:08:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:08:12.907742 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-7b25f516-d024-4f63-859b-3ed1f5593033 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:12.908611 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-7b25f516-d024-4f63-859b-3ed1f5593033 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:12.914894 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7b25f516-d024-4f63-859b-3ed1f5593033 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:12.915367 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7b25f516-d024-4f63-859b-3ed1f5593033 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:12.915801 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7b25f516-d024-4f63-859b-3ed1f5593033 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:12.916486 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7b25f516-d024-4f63-859b-3ed1f5593033 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:12.916938 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7b25f516-d024-4f63-859b-3ed1f5593033 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:12.917397 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7b25f516-d024-4f63-859b-3ed1f5593033 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:13.146809 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-7b25f516-d024-4f63-859b-3ed1f5593033 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:13.155106 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-7b25f516-d024-4f63-859b-3ed1f5593033 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:13.156558 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-7b25f516-d024-4f63-859b-3ed1f5593033 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:13.157744 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-7b25f516-d024-4f63-859b-3ed1f5593033 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1722 microversion: 2.20 time: 0.253849 Sep 04 01:08:13.158365 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 260/535] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:12 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1722 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:13.186174 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-1cf3df31-5e40-461d-b83c-721326d90815 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "0108bcd2-226d-4293-a661-c18cff40bf1a"}} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:08:13.188585 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-1cf3df31-5e40-461d-b83c-721326d90815 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:13.195563 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1cf3df31-5e40-461d-b83c-721326d90815 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:13.196336 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1cf3df31-5e40-461d-b83c-721326d90815 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:13.196777 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1cf3df31-5e40-461d-b83c-721326d90815 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:13.197505 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1cf3df31-5e40-461d-b83c-721326d90815 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:13.198008 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1cf3df31-5e40-461d-b83c-721326d90815 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:13.198436 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1cf3df31-5e40-461d-b83c-721326d90815 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:13.345118 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-82c9415b-6997-439c-b91f-b8e9fc855ed5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:13.346105 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-82c9415b-6997-439c-b91f-b8e9fc855ed5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:13.352793 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-82c9415b-6997-439c-b91f-b8e9fc855ed5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:13.353312 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-82c9415b-6997-439c-b91f-b8e9fc855ed5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:13.366750 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-82c9415b-6997-439c-b91f-b8e9fc855ed5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:13.366750 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-82c9415b-6997-439c-b91f-b8e9fc855ed5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:13.366750 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-82c9415b-6997-439c-b91f-b8e9fc855ed5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:13.366750 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-82c9415b-6997-439c-b91f-b8e9fc855ed5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:13.555443 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-82c9415b-6997-439c-b91f-b8e9fc855ed5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:13.564088 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-82c9415b-6997-439c-b91f-b8e9fc855ed5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1368 microversion: 2.1 time: 0.222867 Sep 04 01:08:13.565004 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 261/536] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:13 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1368 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:08:13.684311 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-3715ba88-088a-4085-a1bc-34ff196c2291 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:13.684975 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-3715ba88-088a-4085-a1bc-34ff196c2291 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:13.691781 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3715ba88-088a-4085-a1bc-34ff196c2291 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:13.692491 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3715ba88-088a-4085-a1bc-34ff196c2291 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:13.692985 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3715ba88-088a-4085-a1bc-34ff196c2291 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:13.693712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3715ba88-088a-4085-a1bc-34ff196c2291 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:13.694156 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3715ba88-088a-4085-a1bc-34ff196c2291 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:13.694572 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3715ba88-088a-4085-a1bc-34ff196c2291 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:13.756356 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-3715ba88-088a-4085-a1bc-34ff196c2291 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:13.764022 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-3715ba88-088a-4085-a1bc-34ff196c2291 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 200 len: 1266 microversion: 2.1 time: 0.084123 Sep 04 01:08:13.764494 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 262/537] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:13 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 1266 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:08:14.266331 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-1cf3df31-5e40-461d-b83c-721326d90815 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "POST /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 1.085122 Sep 04 01:08:14.273135 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 276/538] 10.222.0.42 () {62 vars in 1364 bytes} [Sun Sep 4 01:08:13 2022] POST /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3/os-volume_attachments => generated 188 bytes in 1087 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 04 01:08:14.330303 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-7c02fadc-5b95-4956-afc8-82ceec4d30bf tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:14.330303 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-7c02fadc-5b95-4956-afc8-82ceec4d30bf tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: 2808d334-c39c-4a09-88c7-c1711cce4215] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:14.351172 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7c02fadc-5b95-4956-afc8-82ceec4d30bf tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:14.351853 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7c02fadc-5b95-4956-afc8-82ceec4d30bf tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:14.352379 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7c02fadc-5b95-4956-afc8-82ceec4d30bf tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:14.354847 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7c02fadc-5b95-4956-afc8-82ceec4d30bf tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:14.355408 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7c02fadc-5b95-4956-afc8-82ceec4d30bf tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:14.355932 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7c02fadc-5b95-4956-afc8-82ceec4d30bf tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:14.387083 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-7c02fadc-5b95-4956-afc8-82ceec4d30bf tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: 2808d334-c39c-4a09-88c7-c1711cce4215] Going to try to terminate instance {{(pid=121037) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 04 01:08:14.420709 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-26425525-1084-410d-bd71-a96c2ae73abb tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:08:14.421256 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-26425525-1084-410d-bd71-a96c2ae73abb tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:14.428421 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-26425525-1084-410d-bd71-a96c2ae73abb tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:14.428909 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-26425525-1084-410d-bd71-a96c2ae73abb tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:14.429374 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-26425525-1084-410d-bd71-a96c2ae73abb tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:14.430075 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-26425525-1084-410d-bd71-a96c2ae73abb tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:14.430526 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-26425525-1084-410d-bd71-a96c2ae73abb tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:14.430951 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-26425525-1084-410d-bd71-a96c2ae73abb tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:14.490540 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-26425525-1084-410d-bd71-a96c2ae73abb tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=121036) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Sep 04 01:08:14.654459 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_api:SELECT=100 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:14.764433 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-7c02fadc-5b95-4956-afc8-82ceec4d30bf tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "DELETE /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215" status: 204 len: 0 microversion: 2.1 time: 0.441801 Sep 04 01:08:14.765031 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 263/539] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 4 01:08:14 2022] DELETE /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 04 01:08:14.772754 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-16b8b23a-7a32-457f-8042-3724b7db15ab tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:14.773810 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-16b8b23a-7a32-457f-8042-3724b7db15ab tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:14.779999 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-16b8b23a-7a32-457f-8042-3724b7db15ab tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:14.783467 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-16b8b23a-7a32-457f-8042-3724b7db15ab tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:14.784296 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-16b8b23a-7a32-457f-8042-3724b7db15ab tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:14.785990 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-16b8b23a-7a32-457f-8042-3724b7db15ab tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:14.786623 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-16b8b23a-7a32-457f-8042-3724b7db15ab tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:14.787234 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-16b8b23a-7a32-457f-8042-3724b7db15ab tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:14.837868 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-26425525-1084-410d-bd71-a96c2ae73abb tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "POST /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3/action" status: 202 len: 0 microversion: 2.20 time: 0.419391 Sep 04 01:08:14.838913 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 277/540] 10.222.0.42 () {62 vars in 1319 bytes} [Sun Sep 4 01:08:14 2022] POST /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3/action => generated 0 bytes in 423 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Sep 04 01:08:14.852135 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-48c4c1c7-3a04-42d2-ac77-fd33ea44971b tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:14.853429 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-48c4c1c7-3a04-42d2-ac77-fd33ea44971b tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: 2808d334-c39c-4a09-88c7-c1711cce4215] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:14.859476 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-48c4c1c7-3a04-42d2-ac77-fd33ea44971b tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:14.859961 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-48c4c1c7-3a04-42d2-ac77-fd33ea44971b tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:14.860384 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-48c4c1c7-3a04-42d2-ac77-fd33ea44971b tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:14.861293 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-48c4c1c7-3a04-42d2-ac77-fd33ea44971b tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:14.861894 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-48c4c1c7-3a04-42d2-ac77-fd33ea44971b tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:14.862312 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-48c4c1c7-3a04-42d2-ac77-fd33ea44971b tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:15.022885 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-16b8b23a-7a32-457f-8042-3724b7db15ab tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:15.039118 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-16b8b23a-7a32-457f-8042-3724b7db15ab tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1368 microversion: 2.1 time: 0.265898 Sep 04 01:08:15.039750 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-48c4c1c7-3a04-42d2-ac77-fd33ea44971b tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:15.040959 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 264/541] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:14 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1368 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:15.047201 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-7f10fdb6-5fd9-485c-84f9-dab2b3d07a57 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:15.047968 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-7f10fdb6-5fd9-485c-84f9-dab2b3d07a57 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:15.049000 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-48c4c1c7-3a04-42d2-ac77-fd33ea44971b tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215" status: 200 len: 1513 microversion: 2.1 time: 0.208732 Sep 04 01:08:15.061102 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 278/542] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:14 2022] GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215 => generated 1513 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:15.061102 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-e111c24c-7b0e-4a94-bf3f-2cd1f61dfc99 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:15.061102 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-e111c24c-7b0e-4a94-bf3f-2cd1f61dfc99 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:15.075817 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e111c24c-7b0e-4a94-bf3f-2cd1f61dfc99 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:15.075817 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e111c24c-7b0e-4a94-bf3f-2cd1f61dfc99 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:15.075817 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e111c24c-7b0e-4a94-bf3f-2cd1f61dfc99 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:15.075817 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e111c24c-7b0e-4a94-bf3f-2cd1f61dfc99 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:15.076149 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e111c24c-7b0e-4a94-bf3f-2cd1f61dfc99 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:15.076149 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e111c24c-7b0e-4a94-bf3f-2cd1f61dfc99 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:15.079790 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7f10fdb6-5fd9-485c-84f9-dab2b3d07a57 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:15.084178 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7f10fdb6-5fd9-485c-84f9-dab2b3d07a57 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:15.084178 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7f10fdb6-5fd9-485c-84f9-dab2b3d07a57 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:15.084178 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7f10fdb6-5fd9-485c-84f9-dab2b3d07a57 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:15.085342 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7f10fdb6-5fd9-485c-84f9-dab2b3d07a57 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:15.085342 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7f10fdb6-5fd9-485c-84f9-dab2b3d07a57 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:15.197188 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-7f10fdb6-5fd9-485c-84f9-dab2b3d07a57 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:15.206492 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-7f10fdb6-5fd9-485c-84f9-dab2b3d07a57 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 200 len: 1272 microversion: 2.1 time: 0.164133 Sep 04 01:08:15.223564 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 265/543] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:15 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 1272 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:15.228593 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-04b2e87a-9eb6-4b4b-85c8-4594413e4b28 req-e28bb9d5-7857-4454-b9cd-0d6c30a865a3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "49be5a0b-f748-486e-a546-c07a8e8451a3", "name": "network-vif-unplugged", "status": "completed", "tag": "fcbc60fd-1785-4f94-9436-a520ec06cdaf"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:08:15.239833 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-04b2e87a-9eb6-4b4b-85c8-4594413e4b28 req-e28bb9d5-7857-4454-b9cd-0d6c30a865a3 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:15.240299 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-04b2e87a-9eb6-4b4b-85c8-4594413e4b28 req-e28bb9d5-7857-4454-b9cd-0d6c30a865a3 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:15.240893 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-04b2e87a-9eb6-4b4b-85c8-4594413e4b28 req-e28bb9d5-7857-4454-b9cd-0d6c30a865a3 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:15.244166 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e111c24c-7b0e-4a94-bf3f-2cd1f61dfc99 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:15.254103 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-04b2e87a-9eb6-4b4b-85c8-4594413e4b28 req-e28bb9d5-7857-4454-b9cd-0d6c30a865a3 service nova] Creating event network-vif-unplugged:fcbc60fd-1785-4f94-9436-a520ec06cdaf for instance 49be5a0b-f748-486e-a546-c07a8e8451a3 on hv-cinder-maste Sep 04 01:08:15.254836 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e111c24c-7b0e-4a94-bf3f-2cd1f61dfc99 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:15.256499 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e111c24c-7b0e-4a94-bf3f-2cd1f61dfc99 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:15.257739 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-e111c24c-7b0e-4a94-bf3f-2cd1f61dfc99 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1808 microversion: 2.20 time: 0.205772 Sep 04 01:08:15.258297 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 279/544] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:15 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1808 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:15.260705 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-04b2e87a-9eb6-4b4b-85c8-4594413e4b28 req-e28bb9d5-7857-4454-b9cd-0d6c30a865a3 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.035739 Sep 04 01:08:15.261323 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 266/545] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:08:15 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:08:16.066003 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-5ebd0423-0c33-4ed7-83f9-22f760a919dd tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:16.066680 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-5ebd0423-0c33-4ed7-83f9-22f760a919dd tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:16.072645 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5ebd0423-0c33-4ed7-83f9-22f760a919dd tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:16.073155 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5ebd0423-0c33-4ed7-83f9-22f760a919dd tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:16.073661 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5ebd0423-0c33-4ed7-83f9-22f760a919dd tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:16.074264 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5ebd0423-0c33-4ed7-83f9-22f760a919dd tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:16.074713 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5ebd0423-0c33-4ed7-83f9-22f760a919dd tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:16.075133 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5ebd0423-0c33-4ed7-83f9-22f760a919dd tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:16.086107 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-4272d7ee-79de-45d2-8ef2-49678fd3dbbf tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:16.086989 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-4272d7ee-79de-45d2-8ef2-49678fd3dbbf tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: 2808d334-c39c-4a09-88c7-c1711cce4215] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:16.098629 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4272d7ee-79de-45d2-8ef2-49678fd3dbbf tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:16.100236 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4272d7ee-79de-45d2-8ef2-49678fd3dbbf tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:16.103692 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4272d7ee-79de-45d2-8ef2-49678fd3dbbf tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:16.104496 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4272d7ee-79de-45d2-8ef2-49678fd3dbbf tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:16.105387 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4272d7ee-79de-45d2-8ef2-49678fd3dbbf tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:16.107511 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4272d7ee-79de-45d2-8ef2-49678fd3dbbf tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:16.137657 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:16.212111 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-5ebd0423-0c33-4ed7-83f9-22f760a919dd tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:16.220910 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-5ebd0423-0c33-4ed7-83f9-22f760a919dd tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1559 microversion: 2.1 time: 0.158622 Sep 04 01:08:16.221615 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 280/546] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:16 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1559 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:16.230667 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-ea564838-020b-4472-90f0-c24908f5aac7 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:16.231497 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-ea564838-020b-4472-90f0-c24908f5aac7 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:16.237809 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ea564838-020b-4472-90f0-c24908f5aac7 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:16.242953 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ea564838-020b-4472-90f0-c24908f5aac7 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:16.242953 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ea564838-020b-4472-90f0-c24908f5aac7 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:16.242953 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ea564838-020b-4472-90f0-c24908f5aac7 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:16.242953 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ea564838-020b-4472-90f0-c24908f5aac7 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:16.243254 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ea564838-020b-4472-90f0-c24908f5aac7 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:16.272218 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-4272d7ee-79de-45d2-8ef2-49678fd3dbbf tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:16.280878 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-4272d7ee-79de-45d2-8ef2-49678fd3dbbf tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215" status: 200 len: 1513 microversion: 2.1 time: 0.200427 Sep 04 01:08:16.281478 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 267/547] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:16 2022] GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215 => generated 1513 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:08:16.286437 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-bd1b6799-581e-4ec1-b3af-07d5b3de4dc9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:16.286921 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-bd1b6799-581e-4ec1-b3af-07d5b3de4dc9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:16.292982 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-bd1b6799-581e-4ec1-b3af-07d5b3de4dc9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:16.293695 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-bd1b6799-581e-4ec1-b3af-07d5b3de4dc9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:16.294293 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-bd1b6799-581e-4ec1-b3af-07d5b3de4dc9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:16.295133 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-bd1b6799-581e-4ec1-b3af-07d5b3de4dc9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:16.295730 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-bd1b6799-581e-4ec1-b3af-07d5b3de4dc9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:16.296299 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-bd1b6799-581e-4ec1-b3af-07d5b3de4dc9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:16.377517 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-bd1b6799-581e-4ec1-b3af-07d5b3de4dc9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.094902 Sep 04 01:08:16.378156 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 268/548] 10.222.0.42 () {58 vars in 1275 bytes} [Sun Sep 4 01:08:16 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59/os-interface => generated 285 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:08:16.385507 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-59683a6b-a989-405f-9a9f-cb338dd6f302 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:16.386152 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-59683a6b-a989-405f-9a9f-cb338dd6f302 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:16.393739 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-59683a6b-a989-405f-9a9f-cb338dd6f302 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:16.394845 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-59683a6b-a989-405f-9a9f-cb338dd6f302 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:16.395507 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-59683a6b-a989-405f-9a9f-cb338dd6f302 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:16.396218 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-59683a6b-a989-405f-9a9f-cb338dd6f302 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:16.396669 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-59683a6b-a989-405f-9a9f-cb338dd6f302 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:16.397122 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-59683a6b-a989-405f-9a9f-cb338dd6f302 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:16.404228 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ea564838-020b-4472-90f0-c24908f5aac7 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:16.412399 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-ea564838-020b-4472-90f0-c24908f5aac7 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 200 len: 1314 microversion: 2.1 time: 0.185578 Sep 04 01:08:16.413241 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 281/549] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:16 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 1314 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:08:16.560962 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-59683a6b-a989-405f-9a9f-cb338dd6f302 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:16.570741 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-59683a6b-a989-405f-9a9f-cb338dd6f302 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:16.572195 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-59683a6b-a989-405f-9a9f-cb338dd6f302 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:16.573379 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-59683a6b-a989-405f-9a9f-cb338dd6f302 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1808 microversion: 2.20 time: 0.192974 Sep 04 01:08:16.574185 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 269/550] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:16 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1808 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:17.305196 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-5e4f4ea9-b963-4735-b654-f526cccc4c52 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:17.305809 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-5e4f4ea9-b963-4735-b654-f526cccc4c52 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: 2808d334-c39c-4a09-88c7-c1711cce4215] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:17.307762 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell0:SELECT=2 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:17.308559 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell0:SELECT=8 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:17.313736 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5e4f4ea9-b963-4735-b654-f526cccc4c52 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:17.314217 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5e4f4ea9-b963-4735-b654-f526cccc4c52 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:17.314654 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5e4f4ea9-b963-4735-b654-f526cccc4c52 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:17.315348 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5e4f4ea9-b963-4735-b654-f526cccc4c52 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:17.315795 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5e4f4ea9-b963-4735-b654-f526cccc4c52 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:17.316735 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5e4f4ea9-b963-4735-b654-f526cccc4c52 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:17.435132 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-c5f0b159-92dd-4767-8b25-e5ce59528031 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:17.436158 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-c5f0b159-92dd-4767-8b25-e5ce59528031 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:17.443536 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c5f0b159-92dd-4767-8b25-e5ce59528031 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:17.444061 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c5f0b159-92dd-4767-8b25-e5ce59528031 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:17.444445 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c5f0b159-92dd-4767-8b25-e5ce59528031 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:17.445170 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c5f0b159-92dd-4767-8b25-e5ce59528031 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:17.445615 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c5f0b159-92dd-4767-8b25-e5ce59528031 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:17.446248 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c5f0b159-92dd-4767-8b25-e5ce59528031 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:17.586889 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-c5f0b159-92dd-4767-8b25-e5ce59528031 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:17.595049 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-c5f0b159-92dd-4767-8b25-e5ce59528031 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 200 len: 1314 microversion: 2.1 time: 0.163862 Sep 04 01:08:17.595649 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 270/551] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:17 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 1314 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:17.601190 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-60923f62-adb8-4201-ab73-3d3a76794975 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:17.601802 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-60923f62-adb8-4201-ab73-3d3a76794975 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:17.608093 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-60923f62-adb8-4201-ab73-3d3a76794975 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:17.611555 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-60923f62-adb8-4201-ab73-3d3a76794975 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:17.611555 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-60923f62-adb8-4201-ab73-3d3a76794975 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:17.611555 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-60923f62-adb8-4201-ab73-3d3a76794975 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:17.611555 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-60923f62-adb8-4201-ab73-3d3a76794975 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:17.611892 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-60923f62-adb8-4201-ab73-3d3a76794975 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:17.762150 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-5e4f4ea9-b963-4735-b654-f526cccc4c52 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:17.771605 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-60923f62-adb8-4201-ab73-3d3a76794975 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:17.782267 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-5e4f4ea9-b963-4735-b654-f526cccc4c52 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215" status: 200 len: 1513 microversion: 2.1 time: 0.480927 Sep 04 01:08:17.789286 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 282/552] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:17 2022] GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215 => generated 1513 bytes in 482 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:17.789383 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-60923f62-adb8-4201-ab73-3d3a76794975 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:17.790585 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-60923f62-adb8-4201-ab73-3d3a76794975 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:17.791724 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-60923f62-adb8-4201-ab73-3d3a76794975 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1808 microversion: 2.20 time: 0.195156 Sep 04 01:08:17.792251 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 271/553] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:17 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1808 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:17.813688 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-d2203aba-a01b-49fe-aded-3a1b5c2eb955 req-12d37948-b9a4-48cc-b6ab-675252891cd7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "205e4f5b-43ef-494a-a6ad-64f7445e6b59", "tag": "610a3f9e-176f-46b6-82d0-90b3e3651b7b"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:08:17.823946 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-d2203aba-a01b-49fe-aded-3a1b5c2eb955 req-12d37948-b9a4-48cc-b6ab-675252891cd7 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:17.824587 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-d2203aba-a01b-49fe-aded-3a1b5c2eb955 req-12d37948-b9a4-48cc-b6ab-675252891cd7 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:17.825011 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-d2203aba-a01b-49fe-aded-3a1b5c2eb955 req-12d37948-b9a4-48cc-b6ab-675252891cd7 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:17.839229 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-d2203aba-a01b-49fe-aded-3a1b5c2eb955 req-12d37948-b9a4-48cc-b6ab-675252891cd7 service nova] Creating event network-changed:610a3f9e-176f-46b6-82d0-90b3e3651b7b for instance 205e4f5b-43ef-494a-a6ad-64f7445e6b59 on hv-cinder-maste Sep 04 01:08:17.845029 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-d2203aba-a01b-49fe-aded-3a1b5c2eb955 req-12d37948-b9a4-48cc-b6ab-675252891cd7 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038310 Sep 04 01:08:17.845718 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 283/554] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:08:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:08:18.624000 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-9d0e9934-db76-4c50-bd02-945ada1c5f70 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:18.624620 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-9d0e9934-db76-4c50-bd02-945ada1c5f70 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:18.639366 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9d0e9934-db76-4c50-bd02-945ada1c5f70 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:18.639867 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9d0e9934-db76-4c50-bd02-945ada1c5f70 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:18.640308 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9d0e9934-db76-4c50-bd02-945ada1c5f70 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:18.641038 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9d0e9934-db76-4c50-bd02-945ada1c5f70 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:18.641508 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9d0e9934-db76-4c50-bd02-945ada1c5f70 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:18.641937 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9d0e9934-db76-4c50-bd02-945ada1c5f70 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:18.810596 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-6712c9a3-fef8-4d56-8fe4-5092a1258d8b tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:18.812354 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-6712c9a3-fef8-4d56-8fe4-5092a1258d8b tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: 2808d334-c39c-4a09-88c7-c1711cce4215] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:18.824140 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6712c9a3-fef8-4d56-8fe4-5092a1258d8b tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:18.824704 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6712c9a3-fef8-4d56-8fe4-5092a1258d8b tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:18.825230 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6712c9a3-fef8-4d56-8fe4-5092a1258d8b tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:18.826173 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6712c9a3-fef8-4d56-8fe4-5092a1258d8b tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:18.827026 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6712c9a3-fef8-4d56-8fe4-5092a1258d8b tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:18.829451 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6712c9a3-fef8-4d56-8fe4-5092a1258d8b tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:18.926727 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-9d0e9934-db76-4c50-bd02-945ada1c5f70 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:18.930924 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-6712c9a3-fef8-4d56-8fe4-5092a1258d8b tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:18.937591 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-9d0e9934-db76-4c50-bd02-945ada1c5f70 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 200 len: 1314 microversion: 2.1 time: 0.320722 Sep 04 01:08:18.938805 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 272/555] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:18 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 1314 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:18.946270 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-4364005f-5389-495d-8659-7af9edcd8337 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:18.947095 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-4364005f-5389-495d-8659-7af9edcd8337 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:18.953505 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4364005f-5389-495d-8659-7af9edcd8337 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:18.954200 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4364005f-5389-495d-8659-7af9edcd8337 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:18.954873 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4364005f-5389-495d-8659-7af9edcd8337 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:18.955777 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4364005f-5389-495d-8659-7af9edcd8337 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:18.956445 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4364005f-5389-495d-8659-7af9edcd8337 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:18.958102 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4364005f-5389-495d-8659-7af9edcd8337 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:18.973559 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-6712c9a3-fef8-4d56-8fe4-5092a1258d8b tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215" status: 200 len: 1471 microversion: 2.1 time: 0.169782 Sep 04 01:08:18.981470 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 284/556] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:18 2022] GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215 => generated 1471 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:18.991441 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-f5bc5ede-0f92-4f06-aa70-4e6dba4f6cb2 req-6f2e98c2-82b8-4790-ab33-cd881d5d59e7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dac07994-d434-4a73-802f-31d1772c646c", "tag": "1f56da33-407e-4160-b5c8-764c4a91f864"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:08:19.009906 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-f5bc5ede-0f92-4f06-aa70-4e6dba4f6cb2 req-6f2e98c2-82b8-4790-ab33-cd881d5d59e7 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:19.010586 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-f5bc5ede-0f92-4f06-aa70-4e6dba4f6cb2 req-6f2e98c2-82b8-4790-ab33-cd881d5d59e7 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:19.011038 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-f5bc5ede-0f92-4f06-aa70-4e6dba4f6cb2 req-6f2e98c2-82b8-4790-ab33-cd881d5d59e7 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:19.022975 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-f5bc5ede-0f92-4f06-aa70-4e6dba4f6cb2 req-6f2e98c2-82b8-4790-ab33-cd881d5d59e7 service nova] Creating event network-changed:1f56da33-407e-4160-b5c8-764c4a91f864 for instance dac07994-d434-4a73-802f-31d1772c646c on hv-cinder-maste Sep 04 01:08:19.028051 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-f5bc5ede-0f92-4f06-aa70-4e6dba4f6cb2 req-6f2e98c2-82b8-4790-ab33-cd881d5d59e7 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.045152 Sep 04 01:08:19.028651 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 285/557] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:08:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:08:19.134155 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-4364005f-5389-495d-8659-7af9edcd8337 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:19.142978 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-4364005f-5389-495d-8659-7af9edcd8337 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:19.145065 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-4364005f-5389-495d-8659-7af9edcd8337 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:19.146257 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-4364005f-5389-495d-8659-7af9edcd8337 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1750 microversion: 2.20 time: 0.206326 Sep 04 01:08:19.146819 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 273/558] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:18 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1750 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:19.963921 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-0156f4a5-c023-45c0-b184-df9a6560bd45 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:19.964621 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-0156f4a5-c023-45c0-b184-df9a6560bd45 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:19.970812 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0156f4a5-c023-45c0-b184-df9a6560bd45 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:19.971297 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0156f4a5-c023-45c0-b184-df9a6560bd45 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:19.971735 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0156f4a5-c023-45c0-b184-df9a6560bd45 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:19.972685 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0156f4a5-c023-45c0-b184-df9a6560bd45 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:19.973192 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0156f4a5-c023-45c0-b184-df9a6560bd45 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:19.973628 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0156f4a5-c023-45c0-b184-df9a6560bd45 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:19.998766 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-1af3bfba-0c7b-4cad-a847-caae4bb7a5ad tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:19.999378 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-1af3bfba-0c7b-4cad-a847-caae4bb7a5ad tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: 2808d334-c39c-4a09-88c7-c1711cce4215] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:20.006531 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1af3bfba-0c7b-4cad-a847-caae4bb7a5ad tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:20.007021 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1af3bfba-0c7b-4cad-a847-caae4bb7a5ad tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:20.007451 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1af3bfba-0c7b-4cad-a847-caae4bb7a5ad tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:20.008142 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1af3bfba-0c7b-4cad-a847-caae4bb7a5ad tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:20.008599 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1af3bfba-0c7b-4cad-a847-caae4bb7a5ad tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:20.009050 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1af3bfba-0c7b-4cad-a847-caae4bb7a5ad tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:20.083449 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-1af3bfba-0c7b-4cad-a847-caae4bb7a5ad tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:20.090424 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-1af3bfba-0c7b-4cad-a847-caae4bb7a5ad tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215" status: 200 len: 1471 microversion: 2.1 time: 0.095455 Sep 04 01:08:20.091024 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 274/559] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:19 2022] GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215 => generated 1471 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:08:20.113622 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-0156f4a5-c023-45c0-b184-df9a6560bd45 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:20.117230 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=91,nova_cell1:INSERT=4,nova_cell1:UPDATE=5 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:20.121647 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-0156f4a5-c023-45c0-b184-df9a6560bd45 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 200 len: 1314 microversion: 2.1 time: 0.162464 Sep 04 01:08:20.122204 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 286/560] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:19 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 1314 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:20.174471 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-c630c40b-d63f-4f31-ae4e-05a2e3b5cb56 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:20.175008 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-c630c40b-d63f-4f31-ae4e-05a2e3b5cb56 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:20.181192 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c630c40b-d63f-4f31-ae4e-05a2e3b5cb56 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:20.181671 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c630c40b-d63f-4f31-ae4e-05a2e3b5cb56 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:20.182100 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c630c40b-d63f-4f31-ae4e-05a2e3b5cb56 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:20.182782 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c630c40b-d63f-4f31-ae4e-05a2e3b5cb56 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:20.183227 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c630c40b-d63f-4f31-ae4e-05a2e3b5cb56 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:20.183648 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c630c40b-d63f-4f31-ae4e-05a2e3b5cb56 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:20.543087 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-c630c40b-d63f-4f31-ae4e-05a2e3b5cb56 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:20.551799 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-c630c40b-d63f-4f31-ae4e-05a2e3b5cb56 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:20.553290 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-c630c40b-d63f-4f31-ae4e-05a2e3b5cb56 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:20.554448 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-c630c40b-d63f-4f31-ae4e-05a2e3b5cb56 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1750 microversion: 2.20 time: 0.383866 Sep 04 01:08:20.557331 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 275/561] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:20 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1750 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:21.046069 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-d2be138d-65aa-4c68-850a-df1744549bba req-1868fa2b-d92a-4f6f-b0e2-39fc462ef231 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2808d334-c39c-4a09-88c7-c1711cce4215", "name": "network-vif-deleted", "tag": "449add0b-213e-49af-b0f2-5656f19d796f"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:08:21.056347 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-d2be138d-65aa-4c68-850a-df1744549bba req-1868fa2b-d92a-4f6f-b0e2-39fc462ef231 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:21.056820 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-d2be138d-65aa-4c68-850a-df1744549bba req-1868fa2b-d92a-4f6f-b0e2-39fc462ef231 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:21.057289 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-d2be138d-65aa-4c68-850a-df1744549bba req-1868fa2b-d92a-4f6f-b0e2-39fc462ef231 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:21.070349 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-d2be138d-65aa-4c68-850a-df1744549bba req-1868fa2b-d92a-4f6f-b0e2-39fc462ef231 service nova] Creating event network-vif-deleted:449add0b-213e-49af-b0f2-5656f19d796f for instance 2808d334-c39c-4a09-88c7-c1711cce4215 on hv-cinder-maste Sep 04 01:08:21.075197 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-d2be138d-65aa-4c68-850a-df1744549bba req-1868fa2b-d92a-4f6f-b0e2-39fc462ef231 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032815 Sep 04 01:08:21.075773 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 287/562] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:08:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:08:21.114217 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-0f9b86db-46dc-40a8-bcc3-b146a8653e75 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:21.114855 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-0f9b86db-46dc-40a8-bcc3-b146a8653e75 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: 2808d334-c39c-4a09-88c7-c1711cce4215] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:21.121311 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0f9b86db-46dc-40a8-bcc3-b146a8653e75 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:21.121791 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0f9b86db-46dc-40a8-bcc3-b146a8653e75 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:21.122226 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0f9b86db-46dc-40a8-bcc3-b146a8653e75 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:21.122938 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0f9b86db-46dc-40a8-bcc3-b146a8653e75 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:21.123388 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0f9b86db-46dc-40a8-bcc3-b146a8653e75 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:21.123810 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0f9b86db-46dc-40a8-bcc3-b146a8653e75 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:21.145002 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-46c0a81a-6334-43a8-bcce-1948f2274c50 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:21.145925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-46c0a81a-6334-43a8-bcce-1948f2274c50 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:21.152812 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-46c0a81a-6334-43a8-bcce-1948f2274c50 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:21.153317 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-46c0a81a-6334-43a8-bcce-1948f2274c50 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:21.153749 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-46c0a81a-6334-43a8-bcce-1948f2274c50 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:21.154584 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-46c0a81a-6334-43a8-bcce-1948f2274c50 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:21.155055 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-46c0a81a-6334-43a8-bcce-1948f2274c50 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:21.155585 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-46c0a81a-6334-43a8-bcce-1948f2274c50 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:21.185251 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-0f9b86db-46dc-40a8-bcc3-b146a8653e75 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:21.193892 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-0f9b86db-46dc-40a8-bcc3-b146a8653e75 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215" status: 200 len: 1471 microversion: 2.1 time: 0.083493 Sep 04 01:08:21.194473 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 276/563] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:21 2022] GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215 => generated 1471 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:21.282408 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-46c0a81a-6334-43a8-bcce-1948f2274c50 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:21.290250 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-46c0a81a-6334-43a8-bcce-1948f2274c50 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 200 len: 1314 microversion: 2.1 time: 0.150147 Sep 04 01:08:21.290824 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 288/564] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:21 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 1314 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:08:21.582120 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-ea403acb-ef23-4174-9878-7d6ce838c554 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:21.582906 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-ea403acb-ef23-4174-9878-7d6ce838c554 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:21.588931 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea403acb-ef23-4174-9878-7d6ce838c554 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:21.589660 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea403acb-ef23-4174-9878-7d6ce838c554 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:21.590094 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea403acb-ef23-4174-9878-7d6ce838c554 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:21.590973 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea403acb-ef23-4174-9878-7d6ce838c554 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:21.591427 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea403acb-ef23-4174-9878-7d6ce838c554 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:21.592081 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea403acb-ef23-4174-9878-7d6ce838c554 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:21.801853 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-ea403acb-ef23-4174-9878-7d6ce838c554 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:21.811160 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-ea403acb-ef23-4174-9878-7d6ce838c554 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:21.812856 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-ea403acb-ef23-4174-9878-7d6ce838c554 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:21.814064 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-ea403acb-ef23-4174-9878-7d6ce838c554 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1750 microversion: 2.20 time: 0.236825 Sep 04 01:08:21.814618 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 277/565] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:21 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1750 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:22.221110 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-91ee6603-e0ef-4035-add7-3806f5304ef5 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:22.221110 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-91ee6603-e0ef-4035-add7-3806f5304ef5 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: 2808d334-c39c-4a09-88c7-c1711cce4215] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:22.225708 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-91ee6603-e0ef-4035-add7-3806f5304ef5 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:22.226187 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-91ee6603-e0ef-4035-add7-3806f5304ef5 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:22.226614 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-91ee6603-e0ef-4035-add7-3806f5304ef5 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:22.227310 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-91ee6603-e0ef-4035-add7-3806f5304ef5 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:22.227754 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-91ee6603-e0ef-4035-add7-3806f5304ef5 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:22.228173 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-91ee6603-e0ef-4035-add7-3806f5304ef5 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:22.261291 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-91ee6603-e0ef-4035-add7-3806f5304ef5 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:22.261767 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-91ee6603-e0ef-4035-add7-3806f5304ef5 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:22.262196 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-91ee6603-e0ef-4035-add7-3806f5304ef5 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:22.262636 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.objects.instance [None req-91ee6603-e0ef-4035-add7-3806f5304ef5 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lazy-loading 'fault' on Instance uuid 2808d334-c39c-4a09-88c7-c1711cce4215 {{(pid=121036) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:08:22.306436 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-91ee6603-e0ef-4035-add7-3806f5304ef5 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:22.312596 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-ec2f0142-d9ee-4548-bea9-4c13374ec892 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:22.313293 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-ec2f0142-d9ee-4548-bea9-4c13374ec892 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:22.314078 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-91ee6603-e0ef-4035-add7-3806f5304ef5 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215" status: 200 len: 1308 microversion: 2.1 time: 0.100104 Sep 04 01:08:22.314651 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 289/566] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:22 2022] GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215 => generated 1308 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:22.326848 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ec2f0142-d9ee-4548-bea9-4c13374ec892 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:22.326848 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ec2f0142-d9ee-4548-bea9-4c13374ec892 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:22.326848 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ec2f0142-d9ee-4548-bea9-4c13374ec892 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:22.326848 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ec2f0142-d9ee-4548-bea9-4c13374ec892 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:22.327841 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ec2f0142-d9ee-4548-bea9-4c13374ec892 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:22.327841 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ec2f0142-d9ee-4548-bea9-4c13374ec892 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:22.473155 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-ec2f0142-d9ee-4548-bea9-4c13374ec892 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:22.481230 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-ec2f0142-d9ee-4548-bea9-4c13374ec892 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 200 len: 1314 microversion: 2.1 time: 0.172338 Sep 04 01:08:22.481899 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 278/567] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:22 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 1314 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:08:22.842290 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-2dee6e0e-b5bc-4a04-b349-bd76c843fa15 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:22.842814 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-2dee6e0e-b5bc-4a04-b349-bd76c843fa15 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:22.850529 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2dee6e0e-b5bc-4a04-b349-bd76c843fa15 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:22.851007 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2dee6e0e-b5bc-4a04-b349-bd76c843fa15 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:22.851445 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2dee6e0e-b5bc-4a04-b349-bd76c843fa15 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:22.852133 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2dee6e0e-b5bc-4a04-b349-bd76c843fa15 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:22.852614 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2dee6e0e-b5bc-4a04-b349-bd76c843fa15 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:22.853038 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2dee6e0e-b5bc-4a04-b349-bd76c843fa15 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:22.988150 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-2dee6e0e-b5bc-4a04-b349-bd76c843fa15 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:22.996741 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-2dee6e0e-b5bc-4a04-b349-bd76c843fa15 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:22.998224 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-2dee6e0e-b5bc-4a04-b349-bd76c843fa15 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:22.999448 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-2dee6e0e-b5bc-4a04-b349-bd76c843fa15 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1750 microversion: 2.20 time: 0.161513 Sep 04 01:08:23.000039 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 290/568] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:22 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1750 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:23.342132 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-94dfccfe-11c5-438b-8ff7-e29e526b24b0 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:23.342772 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-94dfccfe-11c5-438b-8ff7-e29e526b24b0 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: 2808d334-c39c-4a09-88c7-c1711cce4215] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:23.349855 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-94dfccfe-11c5-438b-8ff7-e29e526b24b0 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:23.350332 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-94dfccfe-11c5-438b-8ff7-e29e526b24b0 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:23.350767 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-94dfccfe-11c5-438b-8ff7-e29e526b24b0 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:23.351464 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-94dfccfe-11c5-438b-8ff7-e29e526b24b0 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:23.351909 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-94dfccfe-11c5-438b-8ff7-e29e526b24b0 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:23.352328 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-94dfccfe-11c5-438b-8ff7-e29e526b24b0 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:23.382716 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-94dfccfe-11c5-438b-8ff7-e29e526b24b0 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:23.383194 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-94dfccfe-11c5-438b-8ff7-e29e526b24b0 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:23.383629 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-94dfccfe-11c5-438b-8ff7-e29e526b24b0 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:23.384072 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.objects.instance [None req-94dfccfe-11c5-438b-8ff7-e29e526b24b0 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lazy-loading 'fault' on Instance uuid 2808d334-c39c-4a09-88c7-c1711cce4215 {{(pid=121037) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:08:23.431887 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-94dfccfe-11c5-438b-8ff7-e29e526b24b0 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:23.439735 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-94dfccfe-11c5-438b-8ff7-e29e526b24b0 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215" status: 200 len: 1308 microversion: 2.1 time: 0.101587 Sep 04 01:08:23.440302 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 279/569] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:23 2022] GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215 => generated 1308 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:23.504805 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-50b22763-b632-4b30-9b4a-c5aa5072f44a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:23.505436 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-50b22763-b632-4b30-9b4a-c5aa5072f44a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:23.511917 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-50b22763-b632-4b30-9b4a-c5aa5072f44a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:23.512330 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-50b22763-b632-4b30-9b4a-c5aa5072f44a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:23.512760 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-50b22763-b632-4b30-9b4a-c5aa5072f44a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:23.513465 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-50b22763-b632-4b30-9b4a-c5aa5072f44a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:23.513911 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-50b22763-b632-4b30-9b4a-c5aa5072f44a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:23.514329 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-50b22763-b632-4b30-9b4a-c5aa5072f44a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:23.640571 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-50b22763-b632-4b30-9b4a-c5aa5072f44a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:23.648621 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-50b22763-b632-4b30-9b4a-c5aa5072f44a tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 200 len: 1314 microversion: 2.1 time: 0.147635 Sep 04 01:08:23.649209 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 291/570] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:23 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 1314 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:24.027386 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-62f550a6-b735-4c1b-9fd1-019893d57b15 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:24.028090 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-62f550a6-b735-4c1b-9fd1-019893d57b15 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:24.039007 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-62f550a6-b735-4c1b-9fd1-019893d57b15 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:24.039497 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-62f550a6-b735-4c1b-9fd1-019893d57b15 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:24.043424 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-62f550a6-b735-4c1b-9fd1-019893d57b15 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:24.043424 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-62f550a6-b735-4c1b-9fd1-019893d57b15 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:24.043424 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-62f550a6-b735-4c1b-9fd1-019893d57b15 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:24.043424 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-62f550a6-b735-4c1b-9fd1-019893d57b15 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:24.172015 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-62f550a6-b735-4c1b-9fd1-019893d57b15 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:24.180761 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-62f550a6-b735-4c1b-9fd1-019893d57b15 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:24.182232 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-62f550a6-b735-4c1b-9fd1-019893d57b15 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:24.183390 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-62f550a6-b735-4c1b-9fd1-019893d57b15 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1750 microversion: 2.20 time: 0.160946 Sep 04 01:08:24.184064 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 280/571] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:24 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1750 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:08:24.462878 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-579f1776-2345-4487-a958-b6b2b88a14ee tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:24.463629 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-579f1776-2345-4487-a958-b6b2b88a14ee tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: 2808d334-c39c-4a09-88c7-c1711cce4215] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:24.470048 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-579f1776-2345-4487-a958-b6b2b88a14ee tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:24.470535 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-579f1776-2345-4487-a958-b6b2b88a14ee tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:24.471006 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-579f1776-2345-4487-a958-b6b2b88a14ee tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:24.471701 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-579f1776-2345-4487-a958-b6b2b88a14ee tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:24.472184 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-579f1776-2345-4487-a958-b6b2b88a14ee tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:24.472633 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-579f1776-2345-4487-a958-b6b2b88a14ee tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:24.486446 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.wsgi [None req-579f1776-2345-4487-a958-b6b2b88a14ee tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] HTTP exception thrown: Instance 2808d334-c39c-4a09-88c7-c1711cce4215 could not be found. Sep 04 01:08:24.488580 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-579f1776-2345-4487-a958-b6b2b88a14ee tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Request method failure captured: Sep 04 01:08:24.488580 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: request: GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215 HTTP/1.1 Sep 04 01:08:24.488580 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept: application/json Sep 04 01:08:24.488580 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept-Encoding: identity Sep 04 01:08:24.488580 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Connection: close Sep 04 01:08:24.488580 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Length: 0 Sep 04 01:08:24.488580 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Type: application/json Sep 04 01:08:24.488580 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Host: 10.222.0.42 Sep 04 01:08:24.488580 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Openstack-System-Scope: None Sep 04 01:08:24.488580 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: User-Agent: python-urllib3/1.26.12 Sep 04 01:08:24.488580 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Auth-Token: gAAAAABjE_obhYA9_bHoxMJ9YiLa5KIN-wLSWbgj9mawygfSrhTusnb451gJNWZloutBmVTPKPhC2n3ys4WuBDIXKrrGT73hy4If6MlaTXkCBJYE0CckFsyELp2rqoLufNyFZR4gVzjnZc8V2VC3qEMEkJ6VrzeTTKgMMDjHDms0VLLwfh60mtg Sep 04 01:08:24.488580 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Id: None Sep 04 01:08:24.488580 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Name: None Sep 04 01:08:24.488580 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Identity-Status: Confirmed Sep 04 01:08:24.488580 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Is-Admin-Project: True Sep 04 01:08:24.488580 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Id: default Sep 04 01:08:24.488580 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Name: Default Sep 04 01:08:24.488580 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Id: 6071c3a795ba4aa098e3af8cde306d58 Sep 04 01:08:24.488580 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1378097567 Sep 04 01:08:24.488580 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Role: reader,member Sep 04 01:08:24.488580 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Roles: reader,member Sep 04 01:08:24.490143 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6071c3a795ba4aa098e3af8cde306d58"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_6071c3a795ba4aa098e3af8cde306d58", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6071c3a795ba4aa098e3af8cde306d58"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/6071c3a795ba4aa098e3af8cde306d58"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:08:24.490143 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant: tempest-VolumesSnapshotTestJSON-1378097567 Sep 04 01:08:24.490143 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Id: 6071c3a795ba4aa098e3af8cde306d58 Sep 04 01:08:24.490143 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1378097567 Sep 04 01:08:24.490143 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User: tempest-VolumesSnapshotTestJSON-1378097567-project Sep 04 01:08:24.490143 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Id: default Sep 04 01:08:24.490143 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Name: Default Sep 04 01:08:24.490143 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Id: 77de7bb68f3c45d680c8a78dc40df07b Sep 04 01:08:24.490143 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Name: tempest-VolumesSnapshotTestJSON-1378097567-project Sep 04 01:08:24.490143 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: method: > Sep 04 01:08:24.490143 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: exception: Instance 2808d334-c39c-4a09-88c7-c1711cce4215 could not be found. Sep 04 01:08:24.490143 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: action_args: {'id': '2808d334-c39c-4a09-88c7-c1711cce4215'} Sep 04 01:08:24.491374 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:08:24.491374 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:08:24.491374 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:08:24.491374 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:08:24.491374 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:08:24.491374 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:08:24.491374 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:08:24.491374 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:08:24.491374 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:08:24.491374 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:08:24.491374 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2808d334-c39c-4a09-88c7-c1711cce4215 could not be found. Sep 04 01:08:24.491374 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:08:24.491374 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:08:24.491374 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:08:24.491374 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:08:24.491374 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:08:24.491374 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:08:24.491374 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:08:24.491374 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:08:24.491374 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:08:24.492698 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:08:24.492698 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 04 01:08:24.492698 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 04 01:08:24.492698 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:08:24.492698 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:08:24.492698 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:08:24.492698 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:08:24.492698 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2808d334-c39c-4a09-88c7-c1711cce4215 could not be found. Sep 04 01:08:24.492698 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:08:24.492698 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:08:24.492698 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:08:24.492698 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:08:24.492698 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:08:24.492698 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:08:24.492698 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:08:24.492698 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:08:24.492698 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2808d334-c39c-4a09-88c7-c1711cce4215 could not be found. Sep 04 01:08:24.492698 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:08:24.497067 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-579f1776-2345-4487-a958-b6b2b88a14ee tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Returning 404 to user: Instance 2808d334-c39c-4a09-88c7-c1711cce4215 could not be found. {{(pid=121036) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:08:24.497067 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-579f1776-2345-4487-a958-b6b2b88a14ee tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215" status: 404 len: 111 microversion: 2.1 time: 0.031039 Sep 04 01:08:24.497067 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 292/572] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:24 2022] GET /compute/v2.1/servers/2808d334-c39c-4a09-88c7-c1711cce4215 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 04 01:08:24.671770 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-ddfd10aa-522d-4efc-b209-eca1ddb237bc tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:24.672371 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-ddfd10aa-522d-4efc-b209-eca1ddb237bc tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:24.679392 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ddfd10aa-522d-4efc-b209-eca1ddb237bc tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:24.679861 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ddfd10aa-522d-4efc-b209-eca1ddb237bc tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:24.680290 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ddfd10aa-522d-4efc-b209-eca1ddb237bc tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:24.680961 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ddfd10aa-522d-4efc-b209-eca1ddb237bc tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:24.681433 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ddfd10aa-522d-4efc-b209-eca1ddb237bc tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:24.681850 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ddfd10aa-522d-4efc-b209-eca1ddb237bc tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:24.815916 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-ddfd10aa-522d-4efc-b209-eca1ddb237bc tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:24.823605 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-ddfd10aa-522d-4efc-b209-eca1ddb237bc tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 200 len: 1314 microversion: 2.1 time: 0.155672 Sep 04 01:08:24.824193 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 281/573] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:24 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 1314 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:25.209803 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b0130192-e6cd-4147-921a-31a08a604190 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:25.210410 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-b0130192-e6cd-4147-921a-31a08a604190 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:25.216857 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b0130192-e6cd-4147-921a-31a08a604190 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:25.217365 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b0130192-e6cd-4147-921a-31a08a604190 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:25.217805 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b0130192-e6cd-4147-921a-31a08a604190 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:25.218593 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b0130192-e6cd-4147-921a-31a08a604190 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:25.219045 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b0130192-e6cd-4147-921a-31a08a604190 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:25.219523 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b0130192-e6cd-4147-921a-31a08a604190 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:25.358923 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-b0130192-e6cd-4147-921a-31a08a604190 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:25.367765 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-b0130192-e6cd-4147-921a-31a08a604190 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:25.369246 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-b0130192-e6cd-4147-921a-31a08a604190 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:25.370391 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-b0130192-e6cd-4147-921a-31a08a604190 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1806 microversion: 2.20 time: 0.165129 Sep 04 01:08:25.370946 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 293/574] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:25 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1806 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:25.847058 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-b4306fcc-71b9-4592-a1c3-08f7e87de4fd tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:25.847885 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-b4306fcc-71b9-4592-a1c3-08f7e87de4fd tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:25.854367 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b4306fcc-71b9-4592-a1c3-08f7e87de4fd tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:25.854847 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b4306fcc-71b9-4592-a1c3-08f7e87de4fd tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:25.855279 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b4306fcc-71b9-4592-a1c3-08f7e87de4fd tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:25.856380 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b4306fcc-71b9-4592-a1c3-08f7e87de4fd tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:25.856506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b4306fcc-71b9-4592-a1c3-08f7e87de4fd tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:25.856848 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b4306fcc-71b9-4592-a1c3-08f7e87de4fd tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:25.996928 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-b4306fcc-71b9-4592-a1c3-08f7e87de4fd tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:26.004870 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-b4306fcc-71b9-4592-a1c3-08f7e87de4fd tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 200 len: 1314 microversion: 2.1 time: 0.161767 Sep 04 01:08:26.005472 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 282/575] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:25 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 1314 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:26.396177 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-682abf49-44d2-4a50-8f60-abbae63777f4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:26.396790 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-682abf49-44d2-4a50-8f60-abbae63777f4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:26.403386 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-682abf49-44d2-4a50-8f60-abbae63777f4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:26.403917 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-682abf49-44d2-4a50-8f60-abbae63777f4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:26.404292 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-682abf49-44d2-4a50-8f60-abbae63777f4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:26.405001 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-682abf49-44d2-4a50-8f60-abbae63777f4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:26.405458 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-682abf49-44d2-4a50-8f60-abbae63777f4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:26.405885 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-682abf49-44d2-4a50-8f60-abbae63777f4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:26.541203 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-682abf49-44d2-4a50-8f60-abbae63777f4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:26.550063 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-682abf49-44d2-4a50-8f60-abbae63777f4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:26.551523 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-682abf49-44d2-4a50-8f60-abbae63777f4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:26.552678 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-682abf49-44d2-4a50-8f60-abbae63777f4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1806 microversion: 2.20 time: 0.160536 Sep 04 01:08:26.553452 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 294/576] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:26 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1806 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:27.029931 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-0c6c691e-61b8-4cd3-ade3-1c7c2f2115ce tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:27.032266 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-0c6c691e-61b8-4cd3-ade3-1c7c2f2115ce tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:27.052271 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0c6c691e-61b8-4cd3-ade3-1c7c2f2115ce tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:27.053173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0c6c691e-61b8-4cd3-ade3-1c7c2f2115ce tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:27.054634 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0c6c691e-61b8-4cd3-ade3-1c7c2f2115ce tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:27.055363 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0c6c691e-61b8-4cd3-ade3-1c7c2f2115ce tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:27.057811 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0c6c691e-61b8-4cd3-ade3-1c7c2f2115ce tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:27.058308 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0c6c691e-61b8-4cd3-ade3-1c7c2f2115ce tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:27.209589 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-0c6c691e-61b8-4cd3-ade3-1c7c2f2115ce tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:27.217339 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-0c6c691e-61b8-4cd3-ade3-1c7c2f2115ce tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 200 len: 1314 microversion: 2.1 time: 0.193404 Sep 04 01:08:27.217917 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 283/577] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:27 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 1314 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:08:27.578128 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-2423d5a1-0230-4f75-9801-46d99011d0e5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:27.578722 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-2423d5a1-0230-4f75-9801-46d99011d0e5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:27.585519 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2423d5a1-0230-4f75-9801-46d99011d0e5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:27.586351 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2423d5a1-0230-4f75-9801-46d99011d0e5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:27.586786 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2423d5a1-0230-4f75-9801-46d99011d0e5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:27.587469 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2423d5a1-0230-4f75-9801-46d99011d0e5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:27.587914 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2423d5a1-0230-4f75-9801-46d99011d0e5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:27.588357 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2423d5a1-0230-4f75-9801-46d99011d0e5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:27.727599 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-2423d5a1-0230-4f75-9801-46d99011d0e5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:27.736888 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-2423d5a1-0230-4f75-9801-46d99011d0e5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:27.739132 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-2423d5a1-0230-4f75-9801-46d99011d0e5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:27.740602 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-2423d5a1-0230-4f75-9801-46d99011d0e5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1806 microversion: 2.20 time: 0.166401 Sep 04 01:08:27.741179 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 295/578] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:27 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1806 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:28.240068 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-a839d8a1-1889-4bf7-be4f-0b053302f687 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:28.241335 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-a839d8a1-1889-4bf7-be4f-0b053302f687 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:28.248938 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a839d8a1-1889-4bf7-be4f-0b053302f687 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:28.249645 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a839d8a1-1889-4bf7-be4f-0b053302f687 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:28.250290 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a839d8a1-1889-4bf7-be4f-0b053302f687 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:28.251743 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a839d8a1-1889-4bf7-be4f-0b053302f687 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:28.252193 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a839d8a1-1889-4bf7-be4f-0b053302f687 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:28.252759 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a839d8a1-1889-4bf7-be4f-0b053302f687 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:28.258184 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=100 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:28.432841 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-a839d8a1-1889-4bf7-be4f-0b053302f687 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:28.441809 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-a839d8a1-1889-4bf7-be4f-0b053302f687 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 200 len: 1314 microversion: 2.1 time: 0.205678 Sep 04 01:08:28.442617 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 284/579] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:28 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 1314 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:28.772574 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-69c8f130-f64e-4276-8111-5fac85645097 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:28.773865 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-69c8f130-f64e-4276-8111-5fac85645097 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:28.787660 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-69c8f130-f64e-4276-8111-5fac85645097 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:28.789364 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-69c8f130-f64e-4276-8111-5fac85645097 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:28.791724 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-69c8f130-f64e-4276-8111-5fac85645097 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:28.792725 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-69c8f130-f64e-4276-8111-5fac85645097 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:28.793577 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-69c8f130-f64e-4276-8111-5fac85645097 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:28.796835 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-69c8f130-f64e-4276-8111-5fac85645097 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:29.161950 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-69c8f130-f64e-4276-8111-5fac85645097 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:29.183438 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-69c8f130-f64e-4276-8111-5fac85645097 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:29.187931 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-69c8f130-f64e-4276-8111-5fac85645097 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:29.188329 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-69c8f130-f64e-4276-8111-5fac85645097 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1806 microversion: 2.20 time: 0.424908 Sep 04 01:08:29.198752 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 296/580] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:28 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1806 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:29.465826 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-6e6889bc-0609-4aff-8ca3-d20d37561129 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:29.466938 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-6e6889bc-0609-4aff-8ca3-d20d37561129 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:29.481884 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6e6889bc-0609-4aff-8ca3-d20d37561129 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:29.482367 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6e6889bc-0609-4aff-8ca3-d20d37561129 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:29.482796 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6e6889bc-0609-4aff-8ca3-d20d37561129 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:29.483497 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6e6889bc-0609-4aff-8ca3-d20d37561129 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:29.483945 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6e6889bc-0609-4aff-8ca3-d20d37561129 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:29.484624 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6e6889bc-0609-4aff-8ca3-d20d37561129 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:29.744974 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-6e6889bc-0609-4aff-8ca3-d20d37561129 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:29.773764 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-6e6889bc-0609-4aff-8ca3-d20d37561129 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 200 len: 1314 microversion: 2.1 time: 0.311727 Sep 04 01:08:29.789660 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 285/581] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:29 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 1314 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:30.184751 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-62c200ca-4afe-49b9-9019-6c355ed9c63a req-5b3c8d0b-f597-4b65-a448-b280c3859c86 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dac07994-d434-4a73-802f-31d1772c646c", "name": "network-vif-plugged", "status": "completed", "tag": "1f56da33-407e-4160-b5c8-764c4a91f864"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:08:30.207444 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-62c200ca-4afe-49b9-9019-6c355ed9c63a req-5b3c8d0b-f597-4b65-a448-b280c3859c86 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:30.208291 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-62c200ca-4afe-49b9-9019-6c355ed9c63a req-5b3c8d0b-f597-4b65-a448-b280c3859c86 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:30.209048 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-62c200ca-4afe-49b9-9019-6c355ed9c63a req-5b3c8d0b-f597-4b65-a448-b280c3859c86 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:30.215451 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-9fa1c792-88f3-44bf-ac5f-c186e5ac748e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:30.216432 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-9fa1c792-88f3-44bf-ac5f-c186e5ac748e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:30.228985 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9fa1c792-88f3-44bf-ac5f-c186e5ac748e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:30.229741 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9fa1c792-88f3-44bf-ac5f-c186e5ac748e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:30.230467 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9fa1c792-88f3-44bf-ac5f-c186e5ac748e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:30.237909 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9fa1c792-88f3-44bf-ac5f-c186e5ac748e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:30.238055 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-62c200ca-4afe-49b9-9019-6c355ed9c63a req-5b3c8d0b-f597-4b65-a448-b280c3859c86 service nova] Creating event network-vif-plugged:1f56da33-407e-4160-b5c8-764c4a91f864 for instance dac07994-d434-4a73-802f-31d1772c646c on hv-cinder-maste Sep 04 01:08:30.238143 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9fa1c792-88f3-44bf-ac5f-c186e5ac748e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:30.238143 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9fa1c792-88f3-44bf-ac5f-c186e5ac748e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:30.239604 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-62c200ca-4afe-49b9-9019-6c355ed9c63a req-5b3c8d0b-f597-4b65-a448-b280c3859c86 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.057620 Sep 04 01:08:30.240648 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 297/582] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:08:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:08:30.489490 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-9fa1c792-88f3-44bf-ac5f-c186e5ac748e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:30.501345 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-9fa1c792-88f3-44bf-ac5f-c186e5ac748e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:30.504026 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-9fa1c792-88f3-44bf-ac5f-c186e5ac748e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:30.508412 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-9fa1c792-88f3-44bf-ac5f-c186e5ac748e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1806 microversion: 2.20 time: 0.296209 Sep 04 01:08:30.510125 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 286/583] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:30 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1806 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:30.801438 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-02e4723c-f526-4771-9e3f-62ede3f6f74f tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:30.802305 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-02e4723c-f526-4771-9e3f-62ede3f6f74f tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:30.815809 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-02e4723c-f526-4771-9e3f-62ede3f6f74f tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:30.820581 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-02e4723c-f526-4771-9e3f-62ede3f6f74f tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:30.823272 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-02e4723c-f526-4771-9e3f-62ede3f6f74f tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:30.824204 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-02e4723c-f526-4771-9e3f-62ede3f6f74f tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:30.828954 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-02e4723c-f526-4771-9e3f-62ede3f6f74f tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:30.829651 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-02e4723c-f526-4771-9e3f-62ede3f6f74f tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:31.060199 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-02e4723c-f526-4771-9e3f-62ede3f6f74f tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:31.074228 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-02e4723c-f526-4771-9e3f-62ede3f6f74f tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 200 len: 1314 microversion: 2.1 time: 0.278335 Sep 04 01:08:31.075287 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 298/584] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:30 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 1314 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:31.536537 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-10c7e125-3ab1-4b08-a495-4bd6d8e8c1a3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:31.538329 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-10c7e125-3ab1-4b08-a495-4bd6d8e8c1a3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:31.546849 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-10c7e125-3ab1-4b08-a495-4bd6d8e8c1a3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:31.547619 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-10c7e125-3ab1-4b08-a495-4bd6d8e8c1a3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:31.548229 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-10c7e125-3ab1-4b08-a495-4bd6d8e8c1a3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:31.549144 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-10c7e125-3ab1-4b08-a495-4bd6d8e8c1a3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:31.549786 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-10c7e125-3ab1-4b08-a495-4bd6d8e8c1a3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:31.550367 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-10c7e125-3ab1-4b08-a495-4bd6d8e8c1a3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:31.760647 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-10c7e125-3ab1-4b08-a495-4bd6d8e8c1a3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:31.769820 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-10c7e125-3ab1-4b08-a495-4bd6d8e8c1a3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:31.771274 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-10c7e125-3ab1-4b08-a495-4bd6d8e8c1a3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:31.772554 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-10c7e125-3ab1-4b08-a495-4bd6d8e8c1a3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1806 microversion: 2.20 time: 0.241028 Sep 04 01:08:31.773131 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 287/585] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:31 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1806 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:08:32.098505 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-3addcafa-52a8-4630-a21f-8faffb615984 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:32.099325 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-3addcafa-52a8-4630-a21f-8faffb615984 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:32.106128 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3addcafa-52a8-4630-a21f-8faffb615984 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:32.106965 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3addcafa-52a8-4630-a21f-8faffb615984 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:32.107681 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3addcafa-52a8-4630-a21f-8faffb615984 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:32.108478 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3addcafa-52a8-4630-a21f-8faffb615984 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:32.109448 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3addcafa-52a8-4630-a21f-8faffb615984 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:32.110094 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3addcafa-52a8-4630-a21f-8faffb615984 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:32.330464 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-3addcafa-52a8-4630-a21f-8faffb615984 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:32.342470 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-3addcafa-52a8-4630-a21f-8faffb615984 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 200 len: 1314 microversion: 2.1 time: 0.247678 Sep 04 01:08:32.343649 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 299/586] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:32 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 1314 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:32.798633 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-d9364d2f-39d6-4d93-8989-0b53cf7b3b34 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:32.800898 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-d9364d2f-39d6-4d93-8989-0b53cf7b3b34 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:32.809108 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d9364d2f-39d6-4d93-8989-0b53cf7b3b34 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:32.809971 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d9364d2f-39d6-4d93-8989-0b53cf7b3b34 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:32.810857 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d9364d2f-39d6-4d93-8989-0b53cf7b3b34 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:32.811793 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d9364d2f-39d6-4d93-8989-0b53cf7b3b34 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:32.812822 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d9364d2f-39d6-4d93-8989-0b53cf7b3b34 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:32.813552 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d9364d2f-39d6-4d93-8989-0b53cf7b3b34 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:33.046537 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-d9364d2f-39d6-4d93-8989-0b53cf7b3b34 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:33.057314 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-d9364d2f-39d6-4d93-8989-0b53cf7b3b34 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:33.059178 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-d9364d2f-39d6-4d93-8989-0b53cf7b3b34 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:33.060654 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-d9364d2f-39d6-4d93-8989-0b53cf7b3b34 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1806 microversion: 2.20 time: 0.266766 Sep 04 01:08:33.061797 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 288/587] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:32 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1806 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:33.374493 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-c1aa4076-9e12-4dc1-b084-ee6c76903579 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:33.375847 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-c1aa4076-9e12-4dc1-b084-ee6c76903579 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:33.392542 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c1aa4076-9e12-4dc1-b084-ee6c76903579 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:33.395807 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c1aa4076-9e12-4dc1-b084-ee6c76903579 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:33.396546 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c1aa4076-9e12-4dc1-b084-ee6c76903579 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:33.397988 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c1aa4076-9e12-4dc1-b084-ee6c76903579 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:33.401208 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c1aa4076-9e12-4dc1-b084-ee6c76903579 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:33.402137 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c1aa4076-9e12-4dc1-b084-ee6c76903579 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:34.057045 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-c1aa4076-9e12-4dc1-b084-ee6c76903579 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:34.069195 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-c1aa4076-9e12-4dc1-b084-ee6c76903579 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 200 len: 1314 microversion: 2.1 time: 0.698272 Sep 04 01:08:34.070055 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 300/588] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:33 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 1314 bytes in 701 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:34.089321 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-295545e8-737f-47a8-b73c-7a010dcbe520 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:34.090616 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-295545e8-737f-47a8-b73c-7a010dcbe520 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:34.098467 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-295545e8-737f-47a8-b73c-7a010dcbe520 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:34.099338 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-295545e8-737f-47a8-b73c-7a010dcbe520 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:34.099785 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-295545e8-737f-47a8-b73c-7a010dcbe520 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:34.100503 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-295545e8-737f-47a8-b73c-7a010dcbe520 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:34.100960 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-295545e8-737f-47a8-b73c-7a010dcbe520 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:34.101422 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-295545e8-737f-47a8-b73c-7a010dcbe520 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:34.420254 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-295545e8-737f-47a8-b73c-7a010dcbe520 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:34.439379 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-295545e8-737f-47a8-b73c-7a010dcbe520 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:34.443468 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-295545e8-737f-47a8-b73c-7a010dcbe520 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:34.444222 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-295545e8-737f-47a8-b73c-7a010dcbe520 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1806 microversion: 2.20 time: 0.358752 Sep 04 01:08:34.449817 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 289/589] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:34 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1806 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:35.093123 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-accf2a6a-742a-4827-8a7c-97695d729d74 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:35.093790 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-accf2a6a-742a-4827-8a7c-97695d729d74 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:35.101840 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-accf2a6a-742a-4827-8a7c-97695d729d74 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:35.102418 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-accf2a6a-742a-4827-8a7c-97695d729d74 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:35.102861 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-accf2a6a-742a-4827-8a7c-97695d729d74 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:35.103557 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-accf2a6a-742a-4827-8a7c-97695d729d74 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:35.104010 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-accf2a6a-742a-4827-8a7c-97695d729d74 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:35.104433 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-accf2a6a-742a-4827-8a7c-97695d729d74 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:35.315925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-accf2a6a-742a-4827-8a7c-97695d729d74 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:35.336126 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-accf2a6a-742a-4827-8a7c-97695d729d74 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 200 len: 1314 microversion: 2.1 time: 0.246669 Sep 04 01:08:35.341058 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 301/590] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:35 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 1314 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:35.481307 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-a4496ee6-8432-4c2a-8b5d-6a0a80ad52c5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:35.482704 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-a4496ee6-8432-4c2a-8b5d-6a0a80ad52c5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:35.491793 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a4496ee6-8432-4c2a-8b5d-6a0a80ad52c5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:35.492468 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a4496ee6-8432-4c2a-8b5d-6a0a80ad52c5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:35.493139 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a4496ee6-8432-4c2a-8b5d-6a0a80ad52c5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:35.494039 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a4496ee6-8432-4c2a-8b5d-6a0a80ad52c5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:35.494718 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a4496ee6-8432-4c2a-8b5d-6a0a80ad52c5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:35.495327 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a4496ee6-8432-4c2a-8b5d-6a0a80ad52c5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:35.623705 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-0d123491-a593-4027-adf8-47b9650fe7e9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1709910589", "imageRef": "a11315c3-ce81-478b-91e4-5353f189dfa7", "flavorRef": "452", "networks": [{"uuid": "f5f50aa9-3a0d-4b0d-aee8-3b2aef89717d"}]}} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:08:35.708600 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.network.neutron [None req-0d123491-a593-4027-adf8-47b9650fe7e9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] validate_networks() for [('f5f50aa9-3a0d-4b0d-aee8-3b2aef89717d', None, None, None, None, None)] {{(pid=121036) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 04 01:08:35.787140 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-a4496ee6-8432-4c2a-8b5d-6a0a80ad52c5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:35.802566 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-a4496ee6-8432-4c2a-8b5d-6a0a80ad52c5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:35.805035 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-a4496ee6-8432-4c2a-8b5d-6a0a80ad52c5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:35.810340 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-a4496ee6-8432-4c2a-8b5d-6a0a80ad52c5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1806 microversion: 2.20 time: 0.343388 Sep 04 01:08:35.811999 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 290/591] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:35 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1806 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:36.286912 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-0d123491-a593-4027-adf8-47b9650fe7e9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Flavor limits 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 04 01:08:36.291252 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-0d123491-a593-4027-adf8-47b9650fe7e9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Image limits 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 04 01:08:36.292196 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-0d123491-a593-4027-adf8-47b9650fe7e9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Flavor pref 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 04 01:08:36.292771 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-0d123491-a593-4027-adf8-47b9650fe7e9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Image pref 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 04 01:08:36.293369 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-0d123491-a593-4027-adf8-47b9650fe7e9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 04 01:08:36.307133 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0d123491-a593-4027-adf8-47b9650fe7e9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:36.307787 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0d123491-a593-4027-adf8-47b9650fe7e9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:36.308383 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0d123491-a593-4027-adf8-47b9650fe7e9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:36.309319 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0d123491-a593-4027-adf8-47b9650fe7e9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:36.309952 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0d123491-a593-4027-adf8-47b9650fe7e9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:36.310526 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0d123491-a593-4027-adf8-47b9650fe7e9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:36.334698 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-0d123491-a593-4027-adf8-47b9650fe7e9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Getting quotas for project 6071c3a795ba4aa098e3af8cde306d58. Resources: {'ram', 'instances', 'cores'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 04 01:08:36.340877 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-0d123491-a593-4027-adf8-47b9650fe7e9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Getting quotas for user 77de7bb68f3c45d680c8a78dc40df07b and project 6071c3a795ba4aa098e3af8cde306d58. Resources: {'ram', 'instances', 'cores'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 04 01:08:36.352133 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:36.364213 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-13a52374-6382-4f6e-ac7f-bfb08b7639b4 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:36.364213 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-13a52374-6382-4f6e-ac7f-bfb08b7639b4 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:36.372832 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-0d123491-a593-4027-adf8-47b9650fe7e9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Going to run 1 instances... {{(pid=121036) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 04 01:08:36.375406 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-0d123491-a593-4027-adf8-47b9650fe7e9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: a3994b76-60e7-4d9a-a655-2fb09eb68a5d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a11315c3-ce81-478b-91e4-5353f189dfa7',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121036) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 04 01:08:36.387739 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-13a52374-6382-4f6e-ac7f-bfb08b7639b4 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:36.388231 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-13a52374-6382-4f6e-ac7f-bfb08b7639b4 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:36.388662 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-13a52374-6382-4f6e-ac7f-bfb08b7639b4 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:36.396390 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-13a52374-6382-4f6e-ac7f-bfb08b7639b4 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:36.396883 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-13a52374-6382-4f6e-ac7f-bfb08b7639b4 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:36.397345 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-13a52374-6382-4f6e-ac7f-bfb08b7639b4 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:36.556845 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-0d123491-a593-4027-adf8-47b9650fe7e9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.936930 Sep 04 01:08:36.564356 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 302/592] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 4 01:08:35 2022] POST /compute/v2.1/servers => generated 380 bytes in 945 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 04 01:08:36.585293 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-130cb869-704c-41d2-8f57-0f424a71ae86 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:36.586278 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-130cb869-704c-41d2-8f57-0f424a71ae86 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: a3994b76-60e7-4d9a-a655-2fb09eb68a5d] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:36.656598 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-13a52374-6382-4f6e-ac7f-bfb08b7639b4 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:36.671126 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-13a52374-6382-4f6e-ac7f-bfb08b7639b4 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 200 len: 1314 microversion: 2.1 time: 0.313572 Sep 04 01:08:36.685671 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-130cb869-704c-41d2-8f57-0f424a71ae86 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:36.698742 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 291/593] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:36 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 1314 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:08:36.701063 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-130cb869-704c-41d2-8f57-0f424a71ae86 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d" status: 200 len: 1219 microversion: 2.1 time: 0.128355 Sep 04 01:08:36.702163 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 303/594] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:36 2022] GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d => generated 1219 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:36.843625 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-bcd544ef-d798-4750-8af7-9c18bedfe15c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:36.844916 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-bcd544ef-d798-4750-8af7-9c18bedfe15c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:36.856702 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-bcd544ef-d798-4750-8af7-9c18bedfe15c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:36.857653 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-bcd544ef-d798-4750-8af7-9c18bedfe15c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:36.861615 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-bcd544ef-d798-4750-8af7-9c18bedfe15c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:36.863844 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-bcd544ef-d798-4750-8af7-9c18bedfe15c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:36.864774 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-bcd544ef-d798-4750-8af7-9c18bedfe15c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:36.865422 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-bcd544ef-d798-4750-8af7-9c18bedfe15c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:37.189344 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-bcd544ef-d798-4750-8af7-9c18bedfe15c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:37.208499 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-bcd544ef-d798-4750-8af7-9c18bedfe15c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:37.215046 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-bcd544ef-d798-4750-8af7-9c18bedfe15c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:37.217393 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-bcd544ef-d798-4750-8af7-9c18bedfe15c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1806 microversion: 2.20 time: 0.378440 Sep 04 01:08:37.218178 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 292/595] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:36 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1806 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:37.702727 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b327fe5d-f2c0-418e-a9f7-97e9cbd5f831 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:37.703776 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-b327fe5d-f2c0-418e-a9f7-97e9cbd5f831 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:37.712171 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b327fe5d-f2c0-418e-a9f7-97e9cbd5f831 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:37.712655 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b327fe5d-f2c0-418e-a9f7-97e9cbd5f831 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:37.713099 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b327fe5d-f2c0-418e-a9f7-97e9cbd5f831 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:37.713799 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b327fe5d-f2c0-418e-a9f7-97e9cbd5f831 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:37.714246 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b327fe5d-f2c0-418e-a9f7-97e9cbd5f831 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:37.714666 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b327fe5d-f2c0-418e-a9f7-97e9cbd5f831 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:37.725596 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-1459cdeb-cbff-41f3-85f9-6a3b9e4ea974 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:37.726847 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-1459cdeb-cbff-41f3-85f9-6a3b9e4ea974 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: a3994b76-60e7-4d9a-a655-2fb09eb68a5d] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:37.837483 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-1459cdeb-cbff-41f3-85f9-6a3b9e4ea974 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:37.844743 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-1459cdeb-cbff-41f3-85f9-6a3b9e4ea974 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d" status: 200 len: 1219 microversion: 2.1 time: 0.123075 Sep 04 01:08:37.846125 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 293/596] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:37 2022] GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d => generated 1219 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:37.915808 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-b327fe5d-f2c0-418e-a9f7-97e9cbd5f831 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:37.923740 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-b327fe5d-f2c0-418e-a9f7-97e9cbd5f831 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 200 len: 1314 microversion: 2.1 time: 0.227376 Sep 04 01:08:37.924375 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 304/597] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:37 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 1314 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:38.244510 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-23deaba6-e652-4663-8a4c-30e29f1ff8db tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:38.246736 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-23deaba6-e652-4663-8a4c-30e29f1ff8db tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:38.256392 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-23deaba6-e652-4663-8a4c-30e29f1ff8db tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:38.257442 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-23deaba6-e652-4663-8a4c-30e29f1ff8db tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:38.258257 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-23deaba6-e652-4663-8a4c-30e29f1ff8db tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:38.259003 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-23deaba6-e652-4663-8a4c-30e29f1ff8db tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:38.259515 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-23deaba6-e652-4663-8a4c-30e29f1ff8db tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:38.260154 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-23deaba6-e652-4663-8a4c-30e29f1ff8db tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:38.462118 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-23deaba6-e652-4663-8a4c-30e29f1ff8db tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:38.473618 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-23deaba6-e652-4663-8a4c-30e29f1ff8db tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:38.475216 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-23deaba6-e652-4663-8a4c-30e29f1ff8db tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:38.476581 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-23deaba6-e652-4663-8a4c-30e29f1ff8db tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1806 microversion: 2.20 time: 0.236644 Sep 04 01:08:38.477252 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 294/598] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:38 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1806 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:38.868137 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-5df8cd34-7d1a-48c0-903e-3ae0bd118bcb tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:38.869473 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-5df8cd34-7d1a-48c0-903e-3ae0bd118bcb tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: a3994b76-60e7-4d9a-a655-2fb09eb68a5d] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:38.878777 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5df8cd34-7d1a-48c0-903e-3ae0bd118bcb tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:38.879502 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5df8cd34-7d1a-48c0-903e-3ae0bd118bcb tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:38.880055 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5df8cd34-7d1a-48c0-903e-3ae0bd118bcb tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:38.880622 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5df8cd34-7d1a-48c0-903e-3ae0bd118bcb tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:38.881103 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5df8cd34-7d1a-48c0-903e-3ae0bd118bcb tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:38.881571 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5df8cd34-7d1a-48c0-903e-3ae0bd118bcb tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:38.947092 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-a660db17-f73f-4c00-8e01-8a220b578bca tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:38.947725 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-a660db17-f73f-4c00-8e01-8a220b578bca tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:38.949305 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-5df8cd34-7d1a-48c0-903e-3ae0bd118bcb tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:38.957706 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-5df8cd34-7d1a-48c0-903e-3ae0bd118bcb tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d" status: 200 len: 1223 microversion: 2.1 time: 0.093346 Sep 04 01:08:38.958723 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 305/599] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:38 2022] GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d => generated 1223 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:38.972776 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a660db17-f73f-4c00-8e01-8a220b578bca tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:38.975797 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a660db17-f73f-4c00-8e01-8a220b578bca tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:38.976464 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a660db17-f73f-4c00-8e01-8a220b578bca tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:38.977891 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a660db17-f73f-4c00-8e01-8a220b578bca tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:38.978584 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a660db17-f73f-4c00-8e01-8a220b578bca tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:38.979345 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a660db17-f73f-4c00-8e01-8a220b578bca tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:39.309907 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-a660db17-f73f-4c00-8e01-8a220b578bca tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:39.321203 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-a660db17-f73f-4c00-8e01-8a220b578bca tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 200 len: 1314 microversion: 2.1 time: 0.378575 Sep 04 01:08:39.322002 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 295/600] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:38 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 1314 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:08:39.504221 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-2d545d8d-0e3c-4e80-940f-ea15899bb402 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:39.505090 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-2d545d8d-0e3c-4e80-940f-ea15899bb402 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:39.513465 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2d545d8d-0e3c-4e80-940f-ea15899bb402 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:39.517465 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2d545d8d-0e3c-4e80-940f-ea15899bb402 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:39.517465 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2d545d8d-0e3c-4e80-940f-ea15899bb402 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:39.517465 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2d545d8d-0e3c-4e80-940f-ea15899bb402 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:39.517465 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2d545d8d-0e3c-4e80-940f-ea15899bb402 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:39.517834 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2d545d8d-0e3c-4e80-940f-ea15899bb402 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:39.707250 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-2d545d8d-0e3c-4e80-940f-ea15899bb402 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:39.713443 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=100 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:39.718878 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-2d545d8d-0e3c-4e80-940f-ea15899bb402 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:39.720330 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-2d545d8d-0e3c-4e80-940f-ea15899bb402 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:39.722315 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-2d545d8d-0e3c-4e80-940f-ea15899bb402 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1806 microversion: 2.20 time: 0.223172 Sep 04 01:08:39.737636 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 306/601] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:39 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1806 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:39.987254 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-72602054-48ee-41dc-880f-63e4b51c08c3 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:39.987950 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-72602054-48ee-41dc-880f-63e4b51c08c3 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: a3994b76-60e7-4d9a-a655-2fb09eb68a5d] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:39.994418 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-72602054-48ee-41dc-880f-63e4b51c08c3 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:39.994889 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-72602054-48ee-41dc-880f-63e4b51c08c3 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:39.995315 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-72602054-48ee-41dc-880f-63e4b51c08c3 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:39.995993 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-72602054-48ee-41dc-880f-63e4b51c08c3 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:39.996468 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-72602054-48ee-41dc-880f-63e4b51c08c3 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:39.996885 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-72602054-48ee-41dc-880f-63e4b51c08c3 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:40.103121 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-72602054-48ee-41dc-880f-63e4b51c08c3 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:40.113498 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-72602054-48ee-41dc-880f-63e4b51c08c3 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d" status: 200 len: 1271 microversion: 2.1 time: 0.128827 Sep 04 01:08:40.113498 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 296/602] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:39 2022] GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d => generated 1271 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:08:40.353113 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-85569c80-1a89-4d76-a658-21b7094e2556 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:40.353113 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-85569c80-1a89-4d76-a658-21b7094e2556 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:40.366131 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-85569c80-1a89-4d76-a658-21b7094e2556 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:40.366622 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-85569c80-1a89-4d76-a658-21b7094e2556 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:40.367252 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-85569c80-1a89-4d76-a658-21b7094e2556 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:40.368154 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-85569c80-1a89-4d76-a658-21b7094e2556 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:40.368769 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-85569c80-1a89-4d76-a658-21b7094e2556 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:40.369378 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-85569c80-1a89-4d76-a658-21b7094e2556 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:40.548605 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-85569c80-1a89-4d76-a658-21b7094e2556 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:40.562741 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-85569c80-1a89-4d76-a658-21b7094e2556 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 200 len: 1314 microversion: 2.1 time: 0.218198 Sep 04 01:08:40.567938 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 307/603] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:40 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 1314 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:40.748431 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-dc39b08c-f36a-4abc-992e-df5d819de230 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:40.749328 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-dc39b08c-f36a-4abc-992e-df5d819de230 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:40.756669 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-dc39b08c-f36a-4abc-992e-df5d819de230 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:40.757477 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-dc39b08c-f36a-4abc-992e-df5d819de230 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:40.758089 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-dc39b08c-f36a-4abc-992e-df5d819de230 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:40.758965 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-dc39b08c-f36a-4abc-992e-df5d819de230 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:40.759571 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-dc39b08c-f36a-4abc-992e-df5d819de230 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:40.760151 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-dc39b08c-f36a-4abc-992e-df5d819de230 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:40.963709 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-dc39b08c-f36a-4abc-992e-df5d819de230 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:40.977959 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-dc39b08c-f36a-4abc-992e-df5d819de230 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:40.979783 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-dc39b08c-f36a-4abc-992e-df5d819de230 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:40.981060 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-dc39b08c-f36a-4abc-992e-df5d819de230 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1806 microversion: 2.20 time: 0.236659 Sep 04 01:08:40.981763 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 297/604] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:40 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1806 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:41.135973 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-bc9320b1-3bb8-414f-8a19-b5cec9807fd7 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:41.136537 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-bc9320b1-3bb8-414f-8a19-b5cec9807fd7 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: a3994b76-60e7-4d9a-a655-2fb09eb68a5d] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:41.145847 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bc9320b1-3bb8-414f-8a19-b5cec9807fd7 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:41.146347 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bc9320b1-3bb8-414f-8a19-b5cec9807fd7 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:41.151954 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bc9320b1-3bb8-414f-8a19-b5cec9807fd7 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:41.151954 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bc9320b1-3bb8-414f-8a19-b5cec9807fd7 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:41.151954 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bc9320b1-3bb8-414f-8a19-b5cec9807fd7 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:41.151954 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bc9320b1-3bb8-414f-8a19-b5cec9807fd7 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:41.214228 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-bc9320b1-3bb8-414f-8a19-b5cec9807fd7 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:41.222469 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-bc9320b1-3bb8-414f-8a19-b5cec9807fd7 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d" status: 200 len: 1271 microversion: 2.1 time: 0.090424 Sep 04 01:08:41.223394 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 308/605] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:41 2022] GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d => generated 1271 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:41.590853 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-079c6107-5bdf-4806-979d-087699681f9b tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:41.592048 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-079c6107-5bdf-4806-979d-087699681f9b tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:41.598787 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-079c6107-5bdf-4806-979d-087699681f9b tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:41.600294 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-079c6107-5bdf-4806-979d-087699681f9b tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:41.600761 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-079c6107-5bdf-4806-979d-087699681f9b tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:41.601958 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-079c6107-5bdf-4806-979d-087699681f9b tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:41.602421 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-079c6107-5bdf-4806-979d-087699681f9b tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:41.602855 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-079c6107-5bdf-4806-979d-087699681f9b tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:42.009274 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-ff802ab4-d78d-407a-876f-d8b2b5bac3ae tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:42.010036 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-ff802ab4-d78d-407a-876f-d8b2b5bac3ae tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:42.021417 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ff802ab4-d78d-407a-876f-d8b2b5bac3ae tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:42.021922 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ff802ab4-d78d-407a-876f-d8b2b5bac3ae tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:42.022362 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ff802ab4-d78d-407a-876f-d8b2b5bac3ae tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:42.023069 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ff802ab4-d78d-407a-876f-d8b2b5bac3ae tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:42.023524 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ff802ab4-d78d-407a-876f-d8b2b5bac3ae tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:42.023970 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ff802ab4-d78d-407a-876f-d8b2b5bac3ae tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:42.053232 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-079c6107-5bdf-4806-979d-087699681f9b tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:42.067338 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-079c6107-5bdf-4806-979d-087699681f9b tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 200 len: 1497 microversion: 2.1 time: 0.480182 Sep 04 01:08:42.070424 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 298/606] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:41 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 1497 bytes in 484 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:42.184572 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ff802ab4-d78d-407a-876f-d8b2b5bac3ae tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:42.198428 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ff802ab4-d78d-407a-876f-d8b2b5bac3ae tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:42.200732 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ff802ab4-d78d-407a-876f-d8b2b5bac3ae tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:42.201984 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-ff802ab4-d78d-407a-876f-d8b2b5bac3ae tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1806 microversion: 2.20 time: 0.197519 Sep 04 01:08:42.202615 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 309/607] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:42 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1806 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:08:42.248306 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-f5d3f394-4a3a-4b0a-9f00-d1fcfccf284f tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:42.249259 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-f5d3f394-4a3a-4b0a-9f00-d1fcfccf284f tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: a3994b76-60e7-4d9a-a655-2fb09eb68a5d] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:42.256648 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f5d3f394-4a3a-4b0a-9f00-d1fcfccf284f tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:42.257263 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f5d3f394-4a3a-4b0a-9f00-d1fcfccf284f tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:42.257882 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f5d3f394-4a3a-4b0a-9f00-d1fcfccf284f tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:42.258586 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f5d3f394-4a3a-4b0a-9f00-d1fcfccf284f tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:42.259196 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f5d3f394-4a3a-4b0a-9f00-d1fcfccf284f tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:42.259896 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f5d3f394-4a3a-4b0a-9f00-d1fcfccf284f tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:42.407377 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-f5d3f394-4a3a-4b0a-9f00-d1fcfccf284f tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:42.421388 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-f5d3f394-4a3a-4b0a-9f00-d1fcfccf284f tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d" status: 200 len: 1279 microversion: 2.1 time: 0.176910 Sep 04 01:08:42.422805 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 299/608] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:42 2022] GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d => generated 1279 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:43.167553 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-35f9156c-2238-4e57-b483-14237bf8415c tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:43.168100 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-35f9156c-2238-4e57-b483-14237bf8415c tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:43.175286 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-35f9156c-2238-4e57-b483-14237bf8415c tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:43.175776 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-35f9156c-2238-4e57-b483-14237bf8415c tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:43.176212 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-35f9156c-2238-4e57-b483-14237bf8415c tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:43.177015 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-35f9156c-2238-4e57-b483-14237bf8415c tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:43.177667 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-35f9156c-2238-4e57-b483-14237bf8415c tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:43.184004 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-35f9156c-2238-4e57-b483-14237bf8415c tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:43.230248 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-19871d00-7288-4a86-b635-fee7b50702b0 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:43.231117 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-19871d00-7288-4a86-b635-fee7b50702b0 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:43.235894 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-35f9156c-2238-4e57-b483-14237bf8415c tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Going to try to terminate instance {{(pid=121036) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 04 01:08:43.238732 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-19871d00-7288-4a86-b635-fee7b50702b0 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:43.239209 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-19871d00-7288-4a86-b635-fee7b50702b0 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:43.239799 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-19871d00-7288-4a86-b635-fee7b50702b0 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:43.240651 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-19871d00-7288-4a86-b635-fee7b50702b0 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:43.241133 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-19871d00-7288-4a86-b635-fee7b50702b0 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:43.241539 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-19871d00-7288-4a86-b635-fee7b50702b0 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:43.528986 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-19871d00-7288-4a86-b635-fee7b50702b0 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:43.542237 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-19871d00-7288-4a86-b635-fee7b50702b0 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:43.546751 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-19871d00-7288-4a86-b635-fee7b50702b0 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:43.546751 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-19871d00-7288-4a86-b635-fee7b50702b0 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1806 microversion: 2.20 time: 0.321550 Sep 04 01:08:43.546751 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 300/609] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:43 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1806 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:43.568786 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-35f9156c-2238-4e57-b483-14237bf8415c tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "DELETE /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 204 len: 0 microversion: 2.1 time: 0.404973 Sep 04 01:08:43.569655 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 310/610] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 4 01:08:43 2022] DELETE /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 0 bytes in 407 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 04 01:08:43.569959 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-311a00d7-8aba-4735-84f9-91e5aa3abb70 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:43.570847 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-311a00d7-8aba-4735-84f9-91e5aa3abb70 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: a3994b76-60e7-4d9a-a655-2fb09eb68a5d] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:43.578244 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-311a00d7-8aba-4735-84f9-91e5aa3abb70 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:43.581528 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-10c614b9-f4c2-488c-bb43-17e0a325b491 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:43.581693 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-311a00d7-8aba-4735-84f9-91e5aa3abb70 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:43.581781 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-10c614b9-f4c2-488c-bb43-17e0a325b491 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:43.581858 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-311a00d7-8aba-4735-84f9-91e5aa3abb70 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:43.581858 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-311a00d7-8aba-4735-84f9-91e5aa3abb70 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:43.581858 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-311a00d7-8aba-4735-84f9-91e5aa3abb70 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:43.581858 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-311a00d7-8aba-4735-84f9-91e5aa3abb70 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:43.586949 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-10c614b9-f4c2-488c-bb43-17e0a325b491 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:43.588780 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-10c614b9-f4c2-488c-bb43-17e0a325b491 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:43.589457 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-10c614b9-f4c2-488c-bb43-17e0a325b491 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:43.590330 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-10c614b9-f4c2-488c-bb43-17e0a325b491 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:43.591040 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-10c614b9-f4c2-488c-bb43-17e0a325b491 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:43.591613 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-10c614b9-f4c2-488c-bb43-17e0a325b491 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:43.742388 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-10c614b9-f4c2-488c-bb43-17e0a325b491 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:43.751214 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-10c614b9-f4c2-488c-bb43-17e0a325b491 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 200 len: 1503 microversion: 2.1 time: 0.176733 Sep 04 01:08:43.751797 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 311/611] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:43 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 1503 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:08:43.880417 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-311a00d7-8aba-4735-84f9-91e5aa3abb70 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:43.882906 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=100 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:43.894881 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-311a00d7-8aba-4735-84f9-91e5aa3abb70 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d" status: 200 len: 1319 microversion: 2.1 time: 0.347030 Sep 04 01:08:43.895578 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 301/612] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:43 2022] GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d => generated 1319 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:44.572182 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-9c727b1c-480c-4b82-b1f2-8e2142bd7ba2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:44.573618 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-9c727b1c-480c-4b82-b1f2-8e2142bd7ba2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:44.581467 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9c727b1c-480c-4b82-b1f2-8e2142bd7ba2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:44.584852 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9c727b1c-480c-4b82-b1f2-8e2142bd7ba2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:44.584852 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9c727b1c-480c-4b82-b1f2-8e2142bd7ba2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:44.584852 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9c727b1c-480c-4b82-b1f2-8e2142bd7ba2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:44.584852 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9c727b1c-480c-4b82-b1f2-8e2142bd7ba2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:44.585294 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9c727b1c-480c-4b82-b1f2-8e2142bd7ba2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:44.587045 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-a95cccf8-2fbe-40d1-9b4c-531a926de00c tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Action: 'create', calling method: .version_select at 0x7fc10c503430>, body: {"keypair": {"name": "tempest-TestShelveInstance-832617863", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLV7NdjmwVY2WHqQ6RepQCU+94q9SpmHV+DoXoMfZ6f+TmLCqnKL9zYtJLl3V1sclp6qjgkvRXswWwiuzXqHceCJGzZPcTys4/YEFsxV4cEJEQVoQ5eprFzgXPCGUWAYrQ=="}} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:08:44.603315 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-a95cccf8-2fbe-40d1-9b4c-531a926de00c tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Getting quotas for project 735896cd6b3b4c448dd94b3bf7577cb8. Resources: {'key_pairs'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 04 01:08:44.607759 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-a95cccf8-2fbe-40d1-9b4c-531a926de00c tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Getting quotas for user 13182cef4cf8458ab34779ffd7514c6c and project 735896cd6b3b4c448dd94b3bf7577cb8. Resources: {'key_pairs'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 04 01:08:44.709364 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-a95cccf8-2fbe-40d1-9b4c-531a926de00c tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 396 microversion: 2.1 time: 0.291133 Sep 04 01:08:44.710353 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 312/613] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Sep 4 01:08:44 2022] POST /compute/v2.1/os-keypairs => generated 396 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:08:44.734477 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-9c727b1c-480c-4b82-b1f2-8e2142bd7ba2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:44.743500 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-9c727b1c-480c-4b82-b1f2-8e2142bd7ba2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:44.744934 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-9c727b1c-480c-4b82-b1f2-8e2142bd7ba2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:44.746113 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-9c727b1c-480c-4b82-b1f2-8e2142bd7ba2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1806 microversion: 2.20 time: 0.178685 Sep 04 01:08:44.746697 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 302/614] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:44 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1806 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:08:44.775077 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-be358bc2-8a7c-4b2b-9c94-db68a901fbd0 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:44.775693 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-be358bc2-8a7c-4b2b-9c94-db68a901fbd0 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:44.782744 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-be358bc2-8a7c-4b2b-9c94-db68a901fbd0 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:44.783222 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-be358bc2-8a7c-4b2b-9c94-db68a901fbd0 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:44.784364 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-be358bc2-8a7c-4b2b-9c94-db68a901fbd0 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:44.785051 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-be358bc2-8a7c-4b2b-9c94-db68a901fbd0 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:44.785559 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-be358bc2-8a7c-4b2b-9c94-db68a901fbd0 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:44.785977 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-be358bc2-8a7c-4b2b-9c94-db68a901fbd0 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:44.919525 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-773a68ca-8e9e-47b7-9268-cf89b20d38a8 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:44.920281 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-773a68ca-8e9e-47b7-9268-cf89b20d38a8 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: a3994b76-60e7-4d9a-a655-2fb09eb68a5d] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:44.922235 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_api:SELECT=85,nova_api:UPDATE=1 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:44.933585 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-773a68ca-8e9e-47b7-9268-cf89b20d38a8 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:44.934209 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-773a68ca-8e9e-47b7-9268-cf89b20d38a8 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:44.934757 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-773a68ca-8e9e-47b7-9268-cf89b20d38a8 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:44.935565 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-773a68ca-8e9e-47b7-9268-cf89b20d38a8 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:44.936249 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-773a68ca-8e9e-47b7-9268-cf89b20d38a8 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:44.937033 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-773a68ca-8e9e-47b7-9268-cf89b20d38a8 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:44.961529 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-be358bc2-8a7c-4b2b-9c94-db68a901fbd0 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:44.969378 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-be358bc2-8a7c-4b2b-9c94-db68a901fbd0 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 200 len: 1503 microversion: 2.1 time: 0.198239 Sep 04 01:08:44.972335 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 313/615] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:44 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 1503 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:45.074652 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-773a68ca-8e9e-47b7-9268-cf89b20d38a8 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:45.083952 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-773a68ca-8e9e-47b7-9268-cf89b20d38a8 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d" status: 200 len: 1319 microversion: 2.1 time: 0.168416 Sep 04 01:08:45.084543 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 303/616] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:44 2022] GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d => generated 1319 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:45.775320 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-77b6ab7a-ecbe-4bef-ab25-ea813b925d3d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:45.776279 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-77b6ab7a-ecbe-4bef-ab25-ea813b925d3d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:45.790320 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-77b6ab7a-ecbe-4bef-ab25-ea813b925d3d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:45.791345 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-77b6ab7a-ecbe-4bef-ab25-ea813b925d3d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:45.791969 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-77b6ab7a-ecbe-4bef-ab25-ea813b925d3d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:45.792862 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-77b6ab7a-ecbe-4bef-ab25-ea813b925d3d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:45.793509 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-77b6ab7a-ecbe-4bef-ab25-ea813b925d3d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:45.794321 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-77b6ab7a-ecbe-4bef-ab25-ea813b925d3d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:45.992590 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-c674e50e-b1b1-46a4-a14e-d1e0291fb567 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:45.993216 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-c674e50e-b1b1-46a4-a14e-d1e0291fb567 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:45.999817 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c674e50e-b1b1-46a4-a14e-d1e0291fb567 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:46.000355 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c674e50e-b1b1-46a4-a14e-d1e0291fb567 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:46.001779 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c674e50e-b1b1-46a4-a14e-d1e0291fb567 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:46.001779 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c674e50e-b1b1-46a4-a14e-d1e0291fb567 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:46.002929 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c674e50e-b1b1-46a4-a14e-d1e0291fb567 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:46.003453 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c674e50e-b1b1-46a4-a14e-d1e0291fb567 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:46.003620 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-77b6ab7a-ecbe-4bef-ab25-ea813b925d3d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:46.012822 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-77b6ab7a-ecbe-4bef-ab25-ea813b925d3d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:46.014295 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-77b6ab7a-ecbe-4bef-ab25-ea813b925d3d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:46.015493 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-77b6ab7a-ecbe-4bef-ab25-ea813b925d3d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1806 microversion: 2.20 time: 0.247275 Sep 04 01:08:46.016047 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 314/617] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:45 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1806 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:46.109937 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b5d9c518-9823-412a-86c1-314a38d19258 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:46.110670 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-b5d9c518-9823-412a-86c1-314a38d19258 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: a3994b76-60e7-4d9a-a655-2fb09eb68a5d] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:46.116804 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b5d9c518-9823-412a-86c1-314a38d19258 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:46.117415 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b5d9c518-9823-412a-86c1-314a38d19258 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:46.117848 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b5d9c518-9823-412a-86c1-314a38d19258 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:46.118629 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b5d9c518-9823-412a-86c1-314a38d19258 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:46.119172 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b5d9c518-9823-412a-86c1-314a38d19258 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:46.119604 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b5d9c518-9823-412a-86c1-314a38d19258 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:46.201528 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-c674e50e-b1b1-46a4-a14e-d1e0291fb567 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:46.213967 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-c674e50e-b1b1-46a4-a14e-d1e0291fb567 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 200 len: 1503 microversion: 2.1 time: 0.224695 Sep 04 01:08:46.214748 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 304/618] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:45 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 1503 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:46.228703 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-75e182db-b687-4f6d-bd28-fdda58dcd672 req-54c6166f-930a-413d-8800-57aa69b10d6d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a3994b76-60e7-4d9a-a655-2fb09eb68a5d", "tag": "88ae446d-0ce0-4187-b028-5915cabba54c"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:08:46.239659 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-75e182db-b687-4f6d-bd28-fdda58dcd672 req-54c6166f-930a-413d-8800-57aa69b10d6d service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:46.240826 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-75e182db-b687-4f6d-bd28-fdda58dcd672 req-54c6166f-930a-413d-8800-57aa69b10d6d service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:46.241354 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-75e182db-b687-4f6d-bd28-fdda58dcd672 req-54c6166f-930a-413d-8800-57aa69b10d6d service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:46.253119 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-75e182db-b687-4f6d-bd28-fdda58dcd672 req-54c6166f-930a-413d-8800-57aa69b10d6d service nova] Creating event network-changed:88ae446d-0ce0-4187-b028-5915cabba54c for instance a3994b76-60e7-4d9a-a655-2fb09eb68a5d on hv-cinder-maste Sep 04 01:08:46.261188 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-75e182db-b687-4f6d-bd28-fdda58dcd672 req-54c6166f-930a-413d-8800-57aa69b10d6d service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042965 Sep 04 01:08:46.261188 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 305/619] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:08:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:08:46.291624 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-b5d9c518-9823-412a-86c1-314a38d19258 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:46.300124 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-b5d9c518-9823-412a-86c1-314a38d19258 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d" status: 200 len: 1319 microversion: 2.1 time: 0.194936 Sep 04 01:08:46.300770 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 315/620] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:46 2022] GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d => generated 1319 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:08:47.043975 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-73e29943-1a09-4ee1-a35b-d7c3056ca0d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:47.044846 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-73e29943-1a09-4ee1-a35b-d7c3056ca0d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:47.052765 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-73e29943-1a09-4ee1-a35b-d7c3056ca0d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:47.053275 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-73e29943-1a09-4ee1-a35b-d7c3056ca0d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:47.059548 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-73e29943-1a09-4ee1-a35b-d7c3056ca0d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:47.060680 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-73e29943-1a09-4ee1-a35b-d7c3056ca0d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:47.061376 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-73e29943-1a09-4ee1-a35b-d7c3056ca0d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:47.063918 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-73e29943-1a09-4ee1-a35b-d7c3056ca0d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:47.091986 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-0c9847d5-e6e2-4ced-8abc-32744462dddd tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:47.092939 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-0c9847d5-e6e2-4ced-8abc-32744462dddd tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:47.094851 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell0:SELECT=2 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:47.109816 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0c9847d5-e6e2-4ced-8abc-32744462dddd tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:47.111264 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0c9847d5-e6e2-4ced-8abc-32744462dddd tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:47.112822 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0c9847d5-e6e2-4ced-8abc-32744462dddd tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:47.113764 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0c9847d5-e6e2-4ced-8abc-32744462dddd tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:47.114376 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0c9847d5-e6e2-4ced-8abc-32744462dddd tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:47.114979 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0c9847d5-e6e2-4ced-8abc-32744462dddd tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:47.211643 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.objects.instance [None req-0c9847d5-e6e2-4ced-8abc-32744462dddd tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lazy-loading 'services' on Instance uuid d296b21a-c705-4d0b-8402-19a9c6f2e3d2 {{(pid=121036) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:08:47.446281 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-73e29943-1a09-4ee1-a35b-d7c3056ca0d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:47.456653 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-73e29943-1a09-4ee1-a35b-d7c3056ca0d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:47.458416 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-73e29943-1a09-4ee1-a35b-d7c3056ca0d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:47.459815 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-73e29943-1a09-4ee1-a35b-d7c3056ca0d5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1806 microversion: 2.20 time: 0.421962 Sep 04 01:08:47.461025 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 306/621] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:47 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1806 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:47.465903 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-79e3bc1a-57eb-4bcd-a2c9-03edc3517aa3 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:47.466743 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-79e3bc1a-57eb-4bcd-a2c9-03edc3517aa3 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:47.474316 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-79e3bc1a-57eb-4bcd-a2c9-03edc3517aa3 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:47.475131 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-79e3bc1a-57eb-4bcd-a2c9-03edc3517aa3 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:47.475851 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-79e3bc1a-57eb-4bcd-a2c9-03edc3517aa3 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:47.477423 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-79e3bc1a-57eb-4bcd-a2c9-03edc3517aa3 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:47.478057 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-79e3bc1a-57eb-4bcd-a2c9-03edc3517aa3 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:47.478632 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-79e3bc1a-57eb-4bcd-a2c9-03edc3517aa3 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:47.511560 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-0c9847d5-e6e2-4ced-8abc-32744462dddd tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "DELETE /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2/os-volume_attachments/f9d7184f-5fde-40d3-8812-f912f9298d61" status: 202 len: 0 microversion: 2.1 time: 0.425901 Sep 04 01:08:47.512156 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 316/622] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Sep 4 01:08:47 2022] DELETE /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2/os-volume_attachments/f9d7184f-5fde-40d3-8812-f912f9298d61 => generated 0 bytes in 428 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 04 01:08:47.520785 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-220974ee-af1f-4978-aa49-8e79f7157ff3 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:47.532267 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-220974ee-af1f-4978-aa49-8e79f7157ff3 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: a3994b76-60e7-4d9a-a655-2fb09eb68a5d] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:47.539656 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-220974ee-af1f-4978-aa49-8e79f7157ff3 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:47.542116 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-220974ee-af1f-4978-aa49-8e79f7157ff3 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:47.542748 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-220974ee-af1f-4978-aa49-8e79f7157ff3 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:47.543720 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-220974ee-af1f-4978-aa49-8e79f7157ff3 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:47.544183 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-220974ee-af1f-4978-aa49-8e79f7157ff3 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:47.546360 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-220974ee-af1f-4978-aa49-8e79f7157ff3 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:47.573786 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-79e3bc1a-57eb-4bcd-a2c9-03edc3517aa3 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:47.588979 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-79e3bc1a-57eb-4bcd-a2c9-03edc3517aa3 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 200 len: 1461 microversion: 2.1 time: 0.119781 Sep 04 01:08:47.589871 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 307/623] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:47 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 1461 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:47.601693 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-245ae49d-eebd-40b1-a828-44ea8896948d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "10a678c8-9649-469a-b1ef-7e0530ce26b8"}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:08:47.610384 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-245ae49d-eebd-40b1-a828-44ea8896948d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:47.624513 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-245ae49d-eebd-40b1-a828-44ea8896948d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:47.625007 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-245ae49d-eebd-40b1-a828-44ea8896948d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:47.625458 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-245ae49d-eebd-40b1-a828-44ea8896948d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:47.626158 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-245ae49d-eebd-40b1-a828-44ea8896948d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:47.626790 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-245ae49d-eebd-40b1-a828-44ea8896948d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:47.628073 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-245ae49d-eebd-40b1-a828-44ea8896948d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:47.779235 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-220974ee-af1f-4978-aa49-8e79f7157ff3 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:47.788028 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-220974ee-af1f-4978-aa49-8e79f7157ff3 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d" status: 200 len: 1319 microversion: 2.1 time: 0.274521 Sep 04 01:08:47.789869 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 317/624] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:47 2022] GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d => generated 1319 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:47.842170 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell0:SELECT=2 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:48.276057 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-a4cddc58-113e-4d2c-9422-5f5143752097 req-54f254cc-7cd0-4404-9b9f-5b592d852c48 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dac07994-d434-4a73-802f-31d1772c646c", "name": "network-vif-deleted", "tag": "1f56da33-407e-4160-b5c8-764c4a91f864"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:08:48.284921 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-a4cddc58-113e-4d2c-9422-5f5143752097 req-54f254cc-7cd0-4404-9b9f-5b592d852c48 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:48.285432 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-a4cddc58-113e-4d2c-9422-5f5143752097 req-54f254cc-7cd0-4404-9b9f-5b592d852c48 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:48.285858 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-a4cddc58-113e-4d2c-9422-5f5143752097 req-54f254cc-7cd0-4404-9b9f-5b592d852c48 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:48.296658 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-a4cddc58-113e-4d2c-9422-5f5143752097 req-54f254cc-7cd0-4404-9b9f-5b592d852c48 service nova] Creating event network-vif-deleted:1f56da33-407e-4160-b5c8-764c4a91f864 for instance dac07994-d434-4a73-802f-31d1772c646c on hv-cinder-maste Sep 04 01:08:48.301277 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-a4cddc58-113e-4d2c-9422-5f5143752097 req-54f254cc-7cd0-4404-9b9f-5b592d852c48 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030510 Sep 04 01:08:48.302041 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 318/625] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:08:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:08:48.491123 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-018b4999-4250-4d0b-8a29-694e30710691 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:48.491812 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-018b4999-4250-4d0b-8a29-694e30710691 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:48.498051 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-018b4999-4250-4d0b-8a29-694e30710691 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:48.498527 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-018b4999-4250-4d0b-8a29-694e30710691 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:48.498955 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-018b4999-4250-4d0b-8a29-694e30710691 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:48.499632 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-018b4999-4250-4d0b-8a29-694e30710691 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:48.500112 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-018b4999-4250-4d0b-8a29-694e30710691 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:48.500574 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-018b4999-4250-4d0b-8a29-694e30710691 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:48.642560 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-018b4999-4250-4d0b-8a29-694e30710691 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:48.651319 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-018b4999-4250-4d0b-8a29-694e30710691 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:48.652759 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-018b4999-4250-4d0b-8a29-694e30710691 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:48.653929 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-018b4999-4250-4d0b-8a29-694e30710691 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1806 microversion: 2.20 time: 0.167296 Sep 04 01:08:48.654484 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 319/626] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:48 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1806 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:08:48.661878 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-6cbc999e-851f-4c86-8889-46849bf8b8cf tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:48.662594 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-6cbc999e-851f-4c86-8889-46849bf8b8cf tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:48.668567 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6cbc999e-851f-4c86-8889-46849bf8b8cf tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:48.671588 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6cbc999e-851f-4c86-8889-46849bf8b8cf tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:48.671588 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6cbc999e-851f-4c86-8889-46849bf8b8cf tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:48.671588 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6cbc999e-851f-4c86-8889-46849bf8b8cf tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:48.671588 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6cbc999e-851f-4c86-8889-46849bf8b8cf tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:48.671974 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6cbc999e-851f-4c86-8889-46849bf8b8cf tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:48.705439 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6cbc999e-851f-4c86-8889-46849bf8b8cf tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:48.706625 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6cbc999e-851f-4c86-8889-46849bf8b8cf tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:48.707058 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6cbc999e-851f-4c86-8889-46849bf8b8cf tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:48.707506 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.objects.instance [None req-6cbc999e-851f-4c86-8889-46849bf8b8cf tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lazy-loading 'fault' on Instance uuid dac07994-d434-4a73-802f-31d1772c646c {{(pid=121036) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:08:48.748511 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-6cbc999e-851f-4c86-8889-46849bf8b8cf tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:48.757975 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-6cbc999e-851f-4c86-8889-46849bf8b8cf tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 200 len: 1304 microversion: 2.1 time: 0.102308 Sep 04 01:08:48.758669 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 320/627] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:48 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 1304 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:48.812183 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-e0d20be9-d02b-436e-9133-415d345621ab tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:48.812787 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-e0d20be9-d02b-436e-9133-415d345621ab tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: a3994b76-60e7-4d9a-a655-2fb09eb68a5d] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:48.818763 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e0d20be9-d02b-436e-9133-415d345621ab tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:48.819242 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e0d20be9-d02b-436e-9133-415d345621ab tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:48.819679 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e0d20be9-d02b-436e-9133-415d345621ab tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:48.820377 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e0d20be9-d02b-436e-9133-415d345621ab tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:48.820783 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e0d20be9-d02b-436e-9133-415d345621ab tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:48.821234 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e0d20be9-d02b-436e-9133-415d345621ab tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:48.956197 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e0d20be9-d02b-436e-9133-415d345621ab tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:48.964241 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-e0d20be9-d02b-436e-9133-415d345621ab tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d" status: 200 len: 1319 microversion: 2.1 time: 0.156435 Sep 04 01:08:48.964803 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 321/628] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:48 2022] GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d => generated 1319 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:08:49.681127 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-2be6e466-f4a9-4258-8b9d-018103eb438e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:49.681750 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-2be6e466-f4a9-4258-8b9d-018103eb438e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:49.692628 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2be6e466-f4a9-4258-8b9d-018103eb438e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:49.693153 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2be6e466-f4a9-4258-8b9d-018103eb438e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:49.693826 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2be6e466-f4a9-4258-8b9d-018103eb438e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:49.694569 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2be6e466-f4a9-4258-8b9d-018103eb438e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:49.695099 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2be6e466-f4a9-4258-8b9d-018103eb438e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:49.700219 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2be6e466-f4a9-4258-8b9d-018103eb438e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:49.725065 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=1,nova_cell1:INSERT=2 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:49.859182 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-2be6e466-f4a9-4258-8b9d-018103eb438e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:49.868931 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-2be6e466-f4a9-4258-8b9d-018103eb438e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:49.870465 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-2be6e466-f4a9-4258-8b9d-018103eb438e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:49.875347 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-2be6e466-f4a9-4258-8b9d-018103eb438e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1806 microversion: 2.20 time: 0.195297 Sep 04 01:08:49.876050 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 322/629] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:49 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1806 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:08:49.893683 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-c026c1cc-8c7b-477c-bb6b-ada308ea8e42 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:49.894677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-c026c1cc-8c7b-477c-bb6b-ada308ea8e42 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:49.903723 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c026c1cc-8c7b-477c-bb6b-ada308ea8e42 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:49.904230 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c026c1cc-8c7b-477c-bb6b-ada308ea8e42 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:49.904665 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c026c1cc-8c7b-477c-bb6b-ada308ea8e42 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:49.906343 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c026c1cc-8c7b-477c-bb6b-ada308ea8e42 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:49.906789 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c026c1cc-8c7b-477c-bb6b-ada308ea8e42 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:49.909311 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c026c1cc-8c7b-477c-bb6b-ada308ea8e42 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:49.941699 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c026c1cc-8c7b-477c-bb6b-ada308ea8e42 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:49.942515 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c026c1cc-8c7b-477c-bb6b-ada308ea8e42 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:49.943264 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c026c1cc-8c7b-477c-bb6b-ada308ea8e42 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:49.943915 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.objects.instance [None req-c026c1cc-8c7b-477c-bb6b-ada308ea8e42 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lazy-loading 'fault' on Instance uuid dac07994-d434-4a73-802f-31d1772c646c {{(pid=121036) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:08:49.951175 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-245ae49d-eebd-40b1-a828-44ea8896948d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "POST /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 2.359966 Sep 04 01:08:49.952105 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 308/630] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Sep 4 01:08:47 2022] POST /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134/os-volume_attachments => generated 194 bytes in 2361 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:08:49.992278 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-127f28cf-92a1-49be-861e-bf1a43c2acc1 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:49.996373 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-127f28cf-92a1-49be-861e-bf1a43c2acc1 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: a3994b76-60e7-4d9a-a655-2fb09eb68a5d] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:50.000205 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-c026c1cc-8c7b-477c-bb6b-ada308ea8e42 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a6cf41aaddce4a86825bff00477793f9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'bcf6c738f8d842f99d2eec3b896c5b47', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:50.005910 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-127f28cf-92a1-49be-861e-bf1a43c2acc1 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:50.006604 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-127f28cf-92a1-49be-861e-bf1a43c2acc1 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:50.007032 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-127f28cf-92a1-49be-861e-bf1a43c2acc1 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:50.007964 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-127f28cf-92a1-49be-861e-bf1a43c2acc1 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:50.008443 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-127f28cf-92a1-49be-861e-bf1a43c2acc1 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:50.008860 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-127f28cf-92a1-49be-861e-bf1a43c2acc1 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:50.009492 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-c026c1cc-8c7b-477c-bb6b-ada308ea8e42 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 200 len: 1304 microversion: 2.1 time: 0.131644 Sep 04 01:08:50.010262 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 323/631] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:49 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 1304 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:50.406093 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-127f28cf-92a1-49be-861e-bf1a43c2acc1 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:50.414976 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-127f28cf-92a1-49be-861e-bf1a43c2acc1 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d" status: 200 len: 1319 microversion: 2.1 time: 0.430678 Sep 04 01:08:50.415550 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 309/632] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:49 2022] GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d => generated 1319 bytes in 432 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:50.911259 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-fcf180b3-15c9-41e7-89bf-3ea0950747b6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:50.911259 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-fcf180b3-15c9-41e7-89bf-3ea0950747b6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:50.917014 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fcf180b3-15c9-41e7-89bf-3ea0950747b6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:50.917525 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fcf180b3-15c9-41e7-89bf-3ea0950747b6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:50.917957 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fcf180b3-15c9-41e7-89bf-3ea0950747b6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:50.918639 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fcf180b3-15c9-41e7-89bf-3ea0950747b6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:50.919089 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fcf180b3-15c9-41e7-89bf-3ea0950747b6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:50.919512 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fcf180b3-15c9-41e7-89bf-3ea0950747b6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:50.930415 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-a211bf98-f62f-418b-a0f9-3d0c958dab35 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:08:50.937229 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a211bf98-f62f-418b-a0f9-3d0c958dab35 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:50.937702 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a211bf98-f62f-418b-a0f9-3d0c958dab35 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:50.938143 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a211bf98-f62f-418b-a0f9-3d0c958dab35 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:50.962551 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-a211bf98-f62f-418b-a0f9-3d0c958dab35 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Going to try to stop instance {{(pid=121037) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Sep 04 01:08:51.058065 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-fcf180b3-15c9-41e7-89bf-3ea0950747b6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:51.076612 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-fcf180b3-15c9-41e7-89bf-3ea0950747b6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:51.080188 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-fcf180b3-15c9-41e7-89bf-3ea0950747b6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:51.080188 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-fcf180b3-15c9-41e7-89bf-3ea0950747b6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1806 microversion: 2.20 time: 0.174793 Sep 04 01:08:51.082517 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 324/633] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:50 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1806 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:51.087856 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-9d691263-82c7-4599-86d5-cfc7c05ec516 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:51.088578 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-9d691263-82c7-4599-86d5-cfc7c05ec516 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] [instance: dac07994-d434-4a73-802f-31d1772c646c] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:51.099072 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9d691263-82c7-4599-86d5-cfc7c05ec516 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:51.099843 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9d691263-82c7-4599-86d5-cfc7c05ec516 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:51.100384 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9d691263-82c7-4599-86d5-cfc7c05ec516 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:51.101278 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9d691263-82c7-4599-86d5-cfc7c05ec516 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:51.102003 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9d691263-82c7-4599-86d5-cfc7c05ec516 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:51.102874 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9d691263-82c7-4599-86d5-cfc7c05ec516 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:51.118827 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.wsgi [None req-9d691263-82c7-4599-86d5-cfc7c05ec516 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] HTTP exception thrown: Instance dac07994-d434-4a73-802f-31d1772c646c could not be found. Sep 04 01:08:51.120537 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-9d691263-82c7-4599-86d5-cfc7c05ec516 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Request method failure captured: Sep 04 01:08:51.120537 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: request: GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c HTTP/1.1 Sep 04 01:08:51.120537 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept: application/json Sep 04 01:08:51.120537 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept-Encoding: identity Sep 04 01:08:51.120537 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Connection: close Sep 04 01:08:51.120537 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Length: 0 Sep 04 01:08:51.120537 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Type: application/json Sep 04 01:08:51.120537 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Host: 10.222.0.42 Sep 04 01:08:51.120537 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Openstack-System-Scope: None Sep 04 01:08:51.120537 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: User-Agent: python-urllib3/1.26.12 Sep 04 01:08:51.120537 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Auth-Token: gAAAAABjE_pNrD102VA_M1LsqvXxEL4ASLRJkOZs66JwIfhr4LA66jVzzTUo4pav2dTmV9yYU0q13x5ZoPzE1w7gETzsYCVAdHqPe0UHBSCrxKJJDEP1hNwvK2NOJp8ou8Lm331yT-EoM27cMHSV797nDbWpQ9djpIka044FZeyn_90GUP8cR7o Sep 04 01:08:51.120537 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Id: None Sep 04 01:08:51.120537 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Name: None Sep 04 01:08:51.120537 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Identity-Status: Confirmed Sep 04 01:08:51.120537 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Is-Admin-Project: True Sep 04 01:08:51.120537 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Id: default Sep 04 01:08:51.120537 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Name: Default Sep 04 01:08:51.120537 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Id: bcf6c738f8d842f99d2eec3b896c5b47 Sep 04 01:08:51.120537 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Name: tempest-VolumesActionsTest-1122363192 Sep 04 01:08:51.120537 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Role: reader,member Sep 04 01:08:51.120537 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Roles: reader,member Sep 04 01:08:51.124677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/bcf6c738f8d842f99d2eec3b896c5b47"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_bcf6c738f8d842f99d2eec3b896c5b47", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/bcf6c738f8d842f99d2eec3b896c5b47"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/bcf6c738f8d842f99d2eec3b896c5b47"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:08:51.124677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant: tempest-VolumesActionsTest-1122363192 Sep 04 01:08:51.124677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Id: bcf6c738f8d842f99d2eec3b896c5b47 Sep 04 01:08:51.124677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Name: tempest-VolumesActionsTest-1122363192 Sep 04 01:08:51.124677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User: tempest-VolumesActionsTest-1122363192-project Sep 04 01:08:51.124677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Id: default Sep 04 01:08:51.124677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Name: Default Sep 04 01:08:51.124677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Id: a6cf41aaddce4a86825bff00477793f9 Sep 04 01:08:51.124677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Name: tempest-VolumesActionsTest-1122363192-project Sep 04 01:08:51.124677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: method: > Sep 04 01:08:51.124677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: exception: Instance dac07994-d434-4a73-802f-31d1772c646c could not be found. Sep 04 01:08:51.124677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: action_args: {'id': 'dac07994-d434-4a73-802f-31d1772c646c'} Sep 04 01:08:51.125915 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:08:51.125915 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:08:51.125915 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:08:51.125915 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:08:51.125915 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:08:51.125915 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:08:51.125915 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:08:51.125915 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:08:51.125915 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:08:51.125915 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:08:51.125915 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dac07994-d434-4a73-802f-31d1772c646c could not be found. Sep 04 01:08:51.125915 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:08:51.125915 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:08:51.125915 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:08:51.125915 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:08:51.125915 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:08:51.125915 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:08:51.125915 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:08:51.125915 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:08:51.125915 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:08:51.127921 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:08:51.127921 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 04 01:08:51.127921 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 04 01:08:51.127921 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:08:51.127921 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:08:51.127921 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:08:51.127921 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:08:51.127921 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dac07994-d434-4a73-802f-31d1772c646c could not be found. Sep 04 01:08:51.127921 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:08:51.127921 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:08:51.127921 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:08:51.127921 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:08:51.127921 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:08:51.127921 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:08:51.127921 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:08:51.127921 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:08:51.127921 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dac07994-d434-4a73-802f-31d1772c646c could not be found. Sep 04 01:08:51.127921 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:08:51.130089 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-9d691263-82c7-4599-86d5-cfc7c05ec516 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] Returning 404 to user: Instance dac07994-d434-4a73-802f-31d1772c646c could not be found. {{(pid=121036) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:08:51.130089 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-9d691263-82c7-4599-86d5-cfc7c05ec516 tempest-VolumesActionsTest-1122363192 tempest-VolumesActionsTest-1122363192-project] 10.222.0.42 "GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c" status: 404 len: 111 microversion: 2.1 time: 0.039240 Sep 04 01:08:51.131151 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 325/634] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:51 2022] GET /compute/v2.1/servers/dac07994-d434-4a73-802f-31d1772c646c => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 04 01:08:51.150116 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-a211bf98-f62f-418b-a0f9-3d0c958dab35 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "POST /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2/action" status: 202 len: 0 microversion: 2.1 time: 0.223488 Sep 04 01:08:51.150921 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 310/635] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Sep 4 01:08:50 2022] POST /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2/action => generated 0 bytes in 226 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 04 01:08:51.161793 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-aa57e049-2282-4a7a-afca-94de1050f131 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:51.163383 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-aa57e049-2282-4a7a-afca-94de1050f131 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:51.170734 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-aa57e049-2282-4a7a-afca-94de1050f131 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:51.171440 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-aa57e049-2282-4a7a-afca-94de1050f131 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:51.172131 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-aa57e049-2282-4a7a-afca-94de1050f131 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:51.173047 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-aa57e049-2282-4a7a-afca-94de1050f131 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:51.173605 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-aa57e049-2282-4a7a-afca-94de1050f131 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:51.174260 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-aa57e049-2282-4a7a-afca-94de1050f131 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:51.330198 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-aa57e049-2282-4a7a-afca-94de1050f131 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:51.341672 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-aa57e049-2282-4a7a-afca-94de1050f131 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1674 microversion: 2.1 time: 0.184313 Sep 04 01:08:51.341672 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 326/636] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:51 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1674 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:51.443558 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-9021a305-e94b-4562-9bd4-e18b296c4242 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:51.444765 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-9021a305-e94b-4562-9bd4-e18b296c4242 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: a3994b76-60e7-4d9a-a655-2fb09eb68a5d] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:51.457105 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9021a305-e94b-4562-9bd4-e18b296c4242 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:51.457105 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9021a305-e94b-4562-9bd4-e18b296c4242 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:51.457105 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9021a305-e94b-4562-9bd4-e18b296c4242 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:51.457105 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9021a305-e94b-4562-9bd4-e18b296c4242 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:51.458007 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9021a305-e94b-4562-9bd4-e18b296c4242 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:51.458007 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9021a305-e94b-4562-9bd4-e18b296c4242 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:51.529542 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-e6ff4182-b0b9-4c76-b979-e62578c7c0dc req-008c2559-ea94-49a8-8e0e-a981b5f422a4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "49be5a0b-f748-486e-a546-c07a8e8451a3", "name": "network-vif-plugged", "status": "completed", "tag": "fcbc60fd-1785-4f94-9436-a520ec06cdaf"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:08:51.546193 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-e6ff4182-b0b9-4c76-b979-e62578c7c0dc req-008c2559-ea94-49a8-8e0e-a981b5f422a4 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:51.549630 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-e6ff4182-b0b9-4c76-b979-e62578c7c0dc req-008c2559-ea94-49a8-8e0e-a981b5f422a4 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:51.549876 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-e6ff4182-b0b9-4c76-b979-e62578c7c0dc req-008c2559-ea94-49a8-8e0e-a981b5f422a4 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:51.562322 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-e6ff4182-b0b9-4c76-b979-e62578c7c0dc req-008c2559-ea94-49a8-8e0e-a981b5f422a4 service nova] Creating event network-vif-plugged:fcbc60fd-1785-4f94-9436-a520ec06cdaf for instance 49be5a0b-f748-486e-a546-c07a8e8451a3 on hv-cinder-maste Sep 04 01:08:51.567920 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-e6ff4182-b0b9-4c76-b979-e62578c7c0dc req-008c2559-ea94-49a8-8e0e-a981b5f422a4 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041905 Sep 04 01:08:51.568958 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 327/637] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:08:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:08:51.608712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-9021a305-e94b-4562-9bd4-e18b296c4242 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:51.618630 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-9021a305-e94b-4562-9bd4-e18b296c4242 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d" status: 200 len: 1319 microversion: 2.1 time: 0.178718 Sep 04 01:08:51.619685 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 311/638] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:51 2022] GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d => generated 1319 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:52.108411 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-ac4205f3-ca87-4022-b890-a26bdb1975c9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:52.109130 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-ac4205f3-ca87-4022-b890-a26bdb1975c9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:52.116650 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ac4205f3-ca87-4022-b890-a26bdb1975c9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:52.117162 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ac4205f3-ca87-4022-b890-a26bdb1975c9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:52.117594 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ac4205f3-ca87-4022-b890-a26bdb1975c9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:52.118277 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ac4205f3-ca87-4022-b890-a26bdb1975c9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:52.118722 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ac4205f3-ca87-4022-b890-a26bdb1975c9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:52.119140 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ac4205f3-ca87-4022-b890-a26bdb1975c9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:52.317583 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ac4205f3-ca87-4022-b890-a26bdb1975c9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:52.326462 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ac4205f3-ca87-4022-b890-a26bdb1975c9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:52.327957 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ac4205f3-ca87-4022-b890-a26bdb1975c9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:52.329153 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-ac4205f3-ca87-4022-b890-a26bdb1975c9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1806 microversion: 2.20 time: 0.225541 Sep 04 01:08:52.329809 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 328/639] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:52 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1806 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:52.365943 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-5c202ccd-9a3f-4311-a15e-15dad1b0dde9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:52.368995 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-5c202ccd-9a3f-4311-a15e-15dad1b0dde9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:52.372407 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5c202ccd-9a3f-4311-a15e-15dad1b0dde9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:52.372886 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5c202ccd-9a3f-4311-a15e-15dad1b0dde9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:52.373346 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5c202ccd-9a3f-4311-a15e-15dad1b0dde9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:52.374032 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5c202ccd-9a3f-4311-a15e-15dad1b0dde9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:52.374477 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5c202ccd-9a3f-4311-a15e-15dad1b0dde9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:52.374898 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5c202ccd-9a3f-4311-a15e-15dad1b0dde9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:52.564227 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-5c202ccd-9a3f-4311-a15e-15dad1b0dde9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:52.575272 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-5c202ccd-9a3f-4311-a15e-15dad1b0dde9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1674 microversion: 2.1 time: 0.213674 Sep 04 01:08:52.575859 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 312/640] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:52 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1674 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:52.645227 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-1865582c-d0e6-4b5e-88a8-615c43d34fb9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:52.645852 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-1865582c-d0e6-4b5e-88a8-615c43d34fb9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: a3994b76-60e7-4d9a-a655-2fb09eb68a5d] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:52.652774 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1865582c-d0e6-4b5e-88a8-615c43d34fb9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:52.653304 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1865582c-d0e6-4b5e-88a8-615c43d34fb9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:52.653766 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1865582c-d0e6-4b5e-88a8-615c43d34fb9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:52.654471 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1865582c-d0e6-4b5e-88a8-615c43d34fb9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:52.661293 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1865582c-d0e6-4b5e-88a8-615c43d34fb9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:52.661293 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1865582c-d0e6-4b5e-88a8-615c43d34fb9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:52.793844 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-1865582c-d0e6-4b5e-88a8-615c43d34fb9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:52.802528 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-1865582c-d0e6-4b5e-88a8-615c43d34fb9 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d" status: 200 len: 1319 microversion: 2.1 time: 0.162099 Sep 04 01:08:52.803387 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 329/641] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:52 2022] GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d => generated 1319 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:08:53.355781 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-58b4a167-89af-4cff-8101-0256b6bc6dd8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:53.356377 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-58b4a167-89af-4cff-8101-0256b6bc6dd8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:53.364196 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-58b4a167-89af-4cff-8101-0256b6bc6dd8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:53.364667 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-58b4a167-89af-4cff-8101-0256b6bc6dd8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:53.365114 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-58b4a167-89af-4cff-8101-0256b6bc6dd8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:53.365794 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-58b4a167-89af-4cff-8101-0256b6bc6dd8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:53.366237 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-58b4a167-89af-4cff-8101-0256b6bc6dd8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:53.366772 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-58b4a167-89af-4cff-8101-0256b6bc6dd8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:53.502404 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-58b4a167-89af-4cff-8101-0256b6bc6dd8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:53.511996 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-58b4a167-89af-4cff-8101-0256b6bc6dd8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:53.513457 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-58b4a167-89af-4cff-8101-0256b6bc6dd8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:53.514641 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-58b4a167-89af-4cff-8101-0256b6bc6dd8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1806 microversion: 2.20 time: 0.162759 Sep 04 01:08:53.515193 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 313/642] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:53 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1806 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:53.601762 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-44fdc55f-08d0-44e2-87a1-b136c7feecbc tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:53.602372 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-44fdc55f-08d0-44e2-87a1-b136c7feecbc tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:53.608815 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-44fdc55f-08d0-44e2-87a1-b136c7feecbc tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:53.609330 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-44fdc55f-08d0-44e2-87a1-b136c7feecbc tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:53.610034 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-44fdc55f-08d0-44e2-87a1-b136c7feecbc tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:53.610724 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-44fdc55f-08d0-44e2-87a1-b136c7feecbc tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:53.611206 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-44fdc55f-08d0-44e2-87a1-b136c7feecbc tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:53.611640 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-44fdc55f-08d0-44e2-87a1-b136c7feecbc tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:53.748370 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-44fdc55f-08d0-44e2-87a1-b136c7feecbc tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:53.756535 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-44fdc55f-08d0-44e2-87a1-b136c7feecbc tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1674 microversion: 2.1 time: 0.158554 Sep 04 01:08:53.757134 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 330/643] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:53 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1674 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:53.827908 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-f967c37c-0ecd-473b-83aa-316c990afc72 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:53.828505 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-f967c37c-0ecd-473b-83aa-316c990afc72 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: a3994b76-60e7-4d9a-a655-2fb09eb68a5d] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:53.835966 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f967c37c-0ecd-473b-83aa-316c990afc72 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:53.836447 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f967c37c-0ecd-473b-83aa-316c990afc72 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:53.836879 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f967c37c-0ecd-473b-83aa-316c990afc72 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:53.837597 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f967c37c-0ecd-473b-83aa-316c990afc72 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:53.838044 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f967c37c-0ecd-473b-83aa-316c990afc72 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:53.838563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f967c37c-0ecd-473b-83aa-316c990afc72 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:53.977616 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-f967c37c-0ecd-473b-83aa-316c990afc72 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:53.986362 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-f967c37c-0ecd-473b-83aa-316c990afc72 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d" status: 200 len: 1319 microversion: 2.1 time: 0.162324 Sep 04 01:08:53.986937 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 314/644] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:53 2022] GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d => generated 1319 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:54.540826 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-9d4ec535-feee-436f-bd89-2f7d7b80c726 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:54.541474 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-9d4ec535-feee-436f-bd89-2f7d7b80c726 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:54.547481 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9d4ec535-feee-436f-bd89-2f7d7b80c726 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:54.547957 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9d4ec535-feee-436f-bd89-2f7d7b80c726 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:54.548552 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9d4ec535-feee-436f-bd89-2f7d7b80c726 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:54.549467 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9d4ec535-feee-436f-bd89-2f7d7b80c726 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:54.549921 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9d4ec535-feee-436f-bd89-2f7d7b80c726 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:54.550606 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9d4ec535-feee-436f-bd89-2f7d7b80c726 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:54.614108 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell0:SELECT=12 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:54.692049 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-9d4ec535-feee-436f-bd89-2f7d7b80c726 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:54.700613 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-9d4ec535-feee-436f-bd89-2f7d7b80c726 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:54.702114 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-9d4ec535-feee-436f-bd89-2f7d7b80c726 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:54.703272 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-9d4ec535-feee-436f-bd89-2f7d7b80c726 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1806 microversion: 2.20 time: 0.166417 Sep 04 01:08:54.704150 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 331/645] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:54 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1806 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:08:54.780674 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-4d881473-7d9f-48a4-b4b6-53a1c7605257 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:54.781314 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-4d881473-7d9f-48a4-b4b6-53a1c7605257 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:54.791070 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4d881473-7d9f-48a4-b4b6-53a1c7605257 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:54.791566 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4d881473-7d9f-48a4-b4b6-53a1c7605257 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:54.792862 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4d881473-7d9f-48a4-b4b6-53a1c7605257 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:54.792862 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4d881473-7d9f-48a4-b4b6-53a1c7605257 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:54.793835 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4d881473-7d9f-48a4-b4b6-53a1c7605257 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:54.794616 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4d881473-7d9f-48a4-b4b6-53a1c7605257 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:54.926530 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-4d881473-7d9f-48a4-b4b6-53a1c7605257 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:54.934374 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-4d881473-7d9f-48a4-b4b6-53a1c7605257 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1674 microversion: 2.1 time: 0.158038 Sep 04 01:08:54.934955 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 315/646] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:54 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1674 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:55.010499 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-a8f16251-37aa-445d-ab8b-f70fbd4e89c5 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:55.011139 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-a8f16251-37aa-445d-ab8b-f70fbd4e89c5 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: a3994b76-60e7-4d9a-a655-2fb09eb68a5d] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:55.017697 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a8f16251-37aa-445d-ab8b-f70fbd4e89c5 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:55.018202 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a8f16251-37aa-445d-ab8b-f70fbd4e89c5 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:55.018650 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a8f16251-37aa-445d-ab8b-f70fbd4e89c5 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:55.020154 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a8f16251-37aa-445d-ab8b-f70fbd4e89c5 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:55.020626 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a8f16251-37aa-445d-ab8b-f70fbd4e89c5 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:55.021053 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a8f16251-37aa-445d-ab8b-f70fbd4e89c5 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:55.159398 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-a8f16251-37aa-445d-ab8b-f70fbd4e89c5 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:55.167757 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-a8f16251-37aa-445d-ab8b-f70fbd4e89c5 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d" status: 200 len: 1319 microversion: 2.1 time: 0.161637 Sep 04 01:08:55.168325 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 332/647] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:55 2022] GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d => generated 1319 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:55.730715 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-9f8f9f46-986e-4bc2-a1a1-5adec8322796 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:55.731231 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-9f8f9f46-986e-4bc2-a1a1-5adec8322796 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:55.738022 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9f8f9f46-986e-4bc2-a1a1-5adec8322796 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:55.738505 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9f8f9f46-986e-4bc2-a1a1-5adec8322796 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:55.739574 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9f8f9f46-986e-4bc2-a1a1-5adec8322796 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:55.740759 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9f8f9f46-986e-4bc2-a1a1-5adec8322796 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:55.741388 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9f8f9f46-986e-4bc2-a1a1-5adec8322796 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:55.741851 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9f8f9f46-986e-4bc2-a1a1-5adec8322796 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:55.877464 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-9f8f9f46-986e-4bc2-a1a1-5adec8322796 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:55.880888 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:55.888138 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-9f8f9f46-986e-4bc2-a1a1-5adec8322796 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:55.890820 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-9f8f9f46-986e-4bc2-a1a1-5adec8322796 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:55.890820 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-9f8f9f46-986e-4bc2-a1a1-5adec8322796 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1806 microversion: 2.20 time: 0.164145 Sep 04 01:08:55.891473 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 316/648] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:55 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1806 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:55.958062 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-96199b5c-6088-4cbb-a647-6668c7ea7232 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:55.958702 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-96199b5c-6088-4cbb-a647-6668c7ea7232 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:55.965350 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-96199b5c-6088-4cbb-a647-6668c7ea7232 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:55.965833 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-96199b5c-6088-4cbb-a647-6668c7ea7232 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:55.966267 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-96199b5c-6088-4cbb-a647-6668c7ea7232 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:55.966955 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-96199b5c-6088-4cbb-a647-6668c7ea7232 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:55.967410 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-96199b5c-6088-4cbb-a647-6668c7ea7232 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:55.967832 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-96199b5c-6088-4cbb-a647-6668c7ea7232 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:56.099866 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-96199b5c-6088-4cbb-a647-6668c7ea7232 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:56.107570 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-96199b5c-6088-4cbb-a647-6668c7ea7232 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1674 microversion: 2.1 time: 0.153513 Sep 04 01:08:56.108126 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 333/649] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:55 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1674 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:56.191596 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-6c54b149-f084-46e6-80e2-7a815ac2ccb6 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:56.192593 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-6c54b149-f084-46e6-80e2-7a815ac2ccb6 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: a3994b76-60e7-4d9a-a655-2fb09eb68a5d] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:56.199612 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6c54b149-f084-46e6-80e2-7a815ac2ccb6 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:56.201740 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6c54b149-f084-46e6-80e2-7a815ac2ccb6 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:56.201740 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6c54b149-f084-46e6-80e2-7a815ac2ccb6 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:56.201740 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6c54b149-f084-46e6-80e2-7a815ac2ccb6 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:56.201740 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6c54b149-f084-46e6-80e2-7a815ac2ccb6 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:56.202094 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6c54b149-f084-46e6-80e2-7a815ac2ccb6 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:56.333140 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-6c54b149-f084-46e6-80e2-7a815ac2ccb6 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:56.343169 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-6c54b149-f084-46e6-80e2-7a815ac2ccb6 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d" status: 200 len: 1319 microversion: 2.1 time: 0.156187 Sep 04 01:08:56.343756 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 317/650] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:56 2022] GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d => generated 1319 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:56.542107 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-7b4854c1-f871-4019-b71e-6dbbdd8865fe req-7f5cec8f-07a3-48d0-8b1e-f1158899e0c3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a3994b76-60e7-4d9a-a655-2fb09eb68a5d", "name": "network-vif-plugged", "status": "completed", "tag": "88ae446d-0ce0-4187-b028-5915cabba54c"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:08:56.551643 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-7b4854c1-f871-4019-b71e-6dbbdd8865fe req-7f5cec8f-07a3-48d0-8b1e-f1158899e0c3 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:56.552459 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-7b4854c1-f871-4019-b71e-6dbbdd8865fe req-7f5cec8f-07a3-48d0-8b1e-f1158899e0c3 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:56.552908 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-7b4854c1-f871-4019-b71e-6dbbdd8865fe req-7f5cec8f-07a3-48d0-8b1e-f1158899e0c3 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:56.563730 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-7b4854c1-f871-4019-b71e-6dbbdd8865fe req-7f5cec8f-07a3-48d0-8b1e-f1158899e0c3 service nova] Creating event network-vif-plugged:88ae446d-0ce0-4187-b028-5915cabba54c for instance a3994b76-60e7-4d9a-a655-2fb09eb68a5d on hv-cinder-maste Sep 04 01:08:56.569331 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-7b4854c1-f871-4019-b71e-6dbbdd8865fe req-7f5cec8f-07a3-48d0-8b1e-f1158899e0c3 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030786 Sep 04 01:08:56.569861 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 334/651] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:08:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:08:56.916235 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-86c2dc55-bd66-4d1a-bfb7-fd1c8c29261e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:56.920085 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-86c2dc55-bd66-4d1a-bfb7-fd1c8c29261e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:56.923188 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-86c2dc55-bd66-4d1a-bfb7-fd1c8c29261e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:56.923642 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-86c2dc55-bd66-4d1a-bfb7-fd1c8c29261e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:56.924069 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-86c2dc55-bd66-4d1a-bfb7-fd1c8c29261e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:56.924744 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-86c2dc55-bd66-4d1a-bfb7-fd1c8c29261e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:56.925234 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-86c2dc55-bd66-4d1a-bfb7-fd1c8c29261e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:56.925651 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-86c2dc55-bd66-4d1a-bfb7-fd1c8c29261e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:57.059905 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-86c2dc55-bd66-4d1a-bfb7-fd1c8c29261e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:57.070475 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-86c2dc55-bd66-4d1a-bfb7-fd1c8c29261e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:57.072000 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-86c2dc55-bd66-4d1a-bfb7-fd1c8c29261e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:57.073215 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-86c2dc55-bd66-4d1a-bfb7-fd1c8c29261e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1806 microversion: 2.20 time: 0.160824 Sep 04 01:08:57.073778 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 318/652] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:56 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1806 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:57.132902 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-7d1774a3-43cd-499e-b5f1-ea2c9d4e5e7e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:57.133558 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-7d1774a3-43cd-499e-b5f1-ea2c9d4e5e7e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:57.140469 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7d1774a3-43cd-499e-b5f1-ea2c9d4e5e7e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:57.140970 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7d1774a3-43cd-499e-b5f1-ea2c9d4e5e7e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:57.141457 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7d1774a3-43cd-499e-b5f1-ea2c9d4e5e7e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:57.142153 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7d1774a3-43cd-499e-b5f1-ea2c9d4e5e7e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:57.142609 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7d1774a3-43cd-499e-b5f1-ea2c9d4e5e7e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:57.143039 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7d1774a3-43cd-499e-b5f1-ea2c9d4e5e7e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:57.275058 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-7d1774a3-43cd-499e-b5f1-ea2c9d4e5e7e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:57.282682 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-7d1774a3-43cd-499e-b5f1-ea2c9d4e5e7e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1674 microversion: 2.1 time: 0.154616 Sep 04 01:08:57.283245 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 335/653] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:57 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1674 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:57.367417 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-914ae043-2f5a-4521-918c-9de1c7d2ead2 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:57.368028 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-914ae043-2f5a-4521-918c-9de1c7d2ead2 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: a3994b76-60e7-4d9a-a655-2fb09eb68a5d] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:57.374165 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-914ae043-2f5a-4521-918c-9de1c7d2ead2 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:57.374645 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-914ae043-2f5a-4521-918c-9de1c7d2ead2 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:57.375073 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-914ae043-2f5a-4521-918c-9de1c7d2ead2 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:57.375760 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-914ae043-2f5a-4521-918c-9de1c7d2ead2 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:57.376201 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-914ae043-2f5a-4521-918c-9de1c7d2ead2 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:57.376616 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-914ae043-2f5a-4521-918c-9de1c7d2ead2 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:57.521600 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-914ae043-2f5a-4521-918c-9de1c7d2ead2 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:57.529515 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-914ae043-2f5a-4521-918c-9de1c7d2ead2 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d" status: 200 len: 1319 microversion: 2.1 time: 0.166778 Sep 04 01:08:57.530642 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 319/654] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:57 2022] GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d => generated 1319 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:58.101445 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-ded52fc6-fa2a-4864-8f01-6d748bbfa397 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:58.102054 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-ded52fc6-fa2a-4864-8f01-6d748bbfa397 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:58.108176 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ded52fc6-fa2a-4864-8f01-6d748bbfa397 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:58.108698 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ded52fc6-fa2a-4864-8f01-6d748bbfa397 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:58.109201 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ded52fc6-fa2a-4864-8f01-6d748bbfa397 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:58.110434 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ded52fc6-fa2a-4864-8f01-6d748bbfa397 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:58.110893 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ded52fc6-fa2a-4864-8f01-6d748bbfa397 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:58.111371 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ded52fc6-fa2a-4864-8f01-6d748bbfa397 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:58.246651 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ded52fc6-fa2a-4864-8f01-6d748bbfa397 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:58.255123 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ded52fc6-fa2a-4864-8f01-6d748bbfa397 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:58.256565 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ded52fc6-fa2a-4864-8f01-6d748bbfa397 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:58.257857 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-ded52fc6-fa2a-4864-8f01-6d748bbfa397 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1806 microversion: 2.20 time: 0.161121 Sep 04 01:08:58.258415 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 336/655] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:58 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1806 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:58.305702 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-113f7410-70ce-424f-93b6-0049b5c6282e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:58.306310 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-113f7410-70ce-424f-93b6-0049b5c6282e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:58.313256 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-113f7410-70ce-424f-93b6-0049b5c6282e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:58.313739 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-113f7410-70ce-424f-93b6-0049b5c6282e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:58.314208 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-113f7410-70ce-424f-93b6-0049b5c6282e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:58.314907 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-113f7410-70ce-424f-93b6-0049b5c6282e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:58.315594 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-113f7410-70ce-424f-93b6-0049b5c6282e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:58.318137 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-113f7410-70ce-424f-93b6-0049b5c6282e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:58.474190 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-113f7410-70ce-424f-93b6-0049b5c6282e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:58.482680 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-113f7410-70ce-424f-93b6-0049b5c6282e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1674 microversion: 2.1 time: 0.181178 Sep 04 01:08:58.483588 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 320/656] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:58 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1674 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:58.560551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-bc8665e6-e3e2-487d-aed3-149ef793d6fd tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:58.561198 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-bc8665e6-e3e2-487d-aed3-149ef793d6fd tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: a3994b76-60e7-4d9a-a655-2fb09eb68a5d] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:58.568863 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bc8665e6-e3e2-487d-aed3-149ef793d6fd tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:58.569407 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bc8665e6-e3e2-487d-aed3-149ef793d6fd tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:58.570520 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bc8665e6-e3e2-487d-aed3-149ef793d6fd tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:58.571194 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bc8665e6-e3e2-487d-aed3-149ef793d6fd tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:58.571650 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bc8665e6-e3e2-487d-aed3-149ef793d6fd tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:58.572187 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bc8665e6-e3e2-487d-aed3-149ef793d6fd tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:58.732948 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-bc8665e6-e3e2-487d-aed3-149ef793d6fd tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:58.742781 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-bc8665e6-e3e2-487d-aed3-149ef793d6fd tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d" status: 200 len: 1319 microversion: 2.1 time: 0.185631 Sep 04 01:08:58.743140 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 337/657] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:58 2022] GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d => generated 1319 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:59.283840 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-4eb3cc69-a296-4868-a801-7f389c9b2c0c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:59.284475 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-4eb3cc69-a296-4868-a801-7f389c9b2c0c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:59.291244 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4eb3cc69-a296-4868-a801-7f389c9b2c0c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:59.291718 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4eb3cc69-a296-4868-a801-7f389c9b2c0c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:59.292213 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4eb3cc69-a296-4868-a801-7f389c9b2c0c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:59.292826 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4eb3cc69-a296-4868-a801-7f389c9b2c0c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:59.293306 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4eb3cc69-a296-4868-a801-7f389c9b2c0c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:59.293725 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4eb3cc69-a296-4868-a801-7f389c9b2c0c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:59.438195 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-4eb3cc69-a296-4868-a801-7f389c9b2c0c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:59.448553 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-4eb3cc69-a296-4868-a801-7f389c9b2c0c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:59.450080 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-4eb3cc69-a296-4868-a801-7f389c9b2c0c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:59.451257 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-4eb3cc69-a296-4868-a801-7f389c9b2c0c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1793 microversion: 2.20 time: 0.171466 Sep 04 01:08:59.451817 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 321/658] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:08:59 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1793 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:08:59.507900 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-fd35363e-d174-4862-9ede-ad6aeb510b73 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:59.508595 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-fd35363e-d174-4862-9ede-ad6aeb510b73 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:59.515538 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fd35363e-d174-4862-9ede-ad6aeb510b73 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:59.516300 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fd35363e-d174-4862-9ede-ad6aeb510b73 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:59.516884 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fd35363e-d174-4862-9ede-ad6aeb510b73 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:59.517762 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fd35363e-d174-4862-9ede-ad6aeb510b73 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:59.518209 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fd35363e-d174-4862-9ede-ad6aeb510b73 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:59.518815 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fd35363e-d174-4862-9ede-ad6aeb510b73 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:59.667687 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-fd35363e-d174-4862-9ede-ad6aeb510b73 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:59.676786 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-fd35363e-d174-4862-9ede-ad6aeb510b73 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1674 microversion: 2.1 time: 0.173268 Sep 04 01:08:59.680906 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 338/659] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:59 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1674 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:08:59.765950 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-3b681afe-c3a7-49b3-89df-c9218cb62f18 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:08:59.766563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-3b681afe-c3a7-49b3-89df-c9218cb62f18 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: a3994b76-60e7-4d9a-a655-2fb09eb68a5d] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:59.776886 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3b681afe-c3a7-49b3-89df-c9218cb62f18 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:59.777607 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3b681afe-c3a7-49b3-89df-c9218cb62f18 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:59.778044 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3b681afe-c3a7-49b3-89df-c9218cb62f18 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:59.778757 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3b681afe-c3a7-49b3-89df-c9218cb62f18 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:59.779208 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3b681afe-c3a7-49b3-89df-c9218cb62f18 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:59.780762 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3b681afe-c3a7-49b3-89df-c9218cb62f18 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:59.831758 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-a947f2ab-a331-47c0-8a22-47e19bdf2c2e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:08:59.834912 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-a947f2ab-a331-47c0-8a22-47e19bdf2c2e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:08:59.838764 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=4,nova_api:UPDATE=1 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:08:59.844186 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a947f2ab-a331-47c0-8a22-47e19bdf2c2e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:59.844885 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a947f2ab-a331-47c0-8a22-47e19bdf2c2e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:59.845970 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a947f2ab-a331-47c0-8a22-47e19bdf2c2e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:59.852985 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a947f2ab-a331-47c0-8a22-47e19bdf2c2e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:08:59.852985 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a947f2ab-a331-47c0-8a22-47e19bdf2c2e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:08:59.852985 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a947f2ab-a331-47c0-8a22-47e19bdf2c2e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:08:59.879004 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.objects.instance [None req-a947f2ab-a331-47c0-8a22-47e19bdf2c2e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lazy-loading 'flavor' on Instance uuid 2570ee6c-f335-41bc-8e67-f89a62b28134 {{(pid=121036) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:08:59.968267 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-3b681afe-c3a7-49b3-89df-c9218cb62f18 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:08:59.978135 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-3b681afe-c3a7-49b3-89df-c9218cb62f18 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d" status: 200 len: 1319 microversion: 2.1 time: 0.215447 Sep 04 01:08:59.995654 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 322/660] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:08:59 2022] GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d => generated 1319 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:00.064224 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:09:00.277909 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-a947f2ab-a331-47c0-8a22-47e19bdf2c2e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "POST /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134/action" status: 200 len: 32 microversion: 2.1 time: 0.449574 Sep 04 01:09:00.283789 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 339/661] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Sep 4 01:08:59 2022] POST /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134/action => generated 32 bytes in 453 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Sep 04 01:09:00.289551 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-81f25b46-93d1-4ca9-a810-ad0c144a172d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:00.290341 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-81f25b46-93d1-4ca9-a810-ad0c144a172d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:00.297222 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-81f25b46-93d1-4ca9-a810-ad0c144a172d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:00.298010 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-81f25b46-93d1-4ca9-a810-ad0c144a172d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:00.298575 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-81f25b46-93d1-4ca9-a810-ad0c144a172d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:00.299400 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-81f25b46-93d1-4ca9-a810-ad0c144a172d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:00.299900 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-81f25b46-93d1-4ca9-a810-ad0c144a172d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:00.300349 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-81f25b46-93d1-4ca9-a810-ad0c144a172d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:00.452744 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-81f25b46-93d1-4ca9-a810-ad0c144a172d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:00.462184 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-81f25b46-93d1-4ca9-a810-ad0c144a172d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1735 microversion: 2.1 time: 0.176858 Sep 04 01:09:00.463389 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 323/662] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:00 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1735 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:00.707044 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b6b51f74-1b05-4827-b60f-dfd3c128a083 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:00.708522 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-b6b51f74-1b05-4827-b60f-dfd3c128a083 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:00.716463 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b6b51f74-1b05-4827-b60f-dfd3c128a083 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:00.716957 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b6b51f74-1b05-4827-b60f-dfd3c128a083 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:00.717437 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b6b51f74-1b05-4827-b60f-dfd3c128a083 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:00.718148 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b6b51f74-1b05-4827-b60f-dfd3c128a083 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:00.718772 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b6b51f74-1b05-4827-b60f-dfd3c128a083 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:00.719395 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b6b51f74-1b05-4827-b60f-dfd3c128a083 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:00.923126 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-b6b51f74-1b05-4827-b60f-dfd3c128a083 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:00.932508 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-b6b51f74-1b05-4827-b60f-dfd3c128a083 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1674 microversion: 2.1 time: 0.234807 Sep 04 01:09:00.933238 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 340/663] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:00 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1674 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:01.006496 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-d2b59dcf-cebb-45af-9016-7705bd2a9b9d tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:01.007180 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-d2b59dcf-cebb-45af-9016-7705bd2a9b9d tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: a3994b76-60e7-4d9a-a655-2fb09eb68a5d] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:01.013886 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d2b59dcf-cebb-45af-9016-7705bd2a9b9d tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:01.014712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d2b59dcf-cebb-45af-9016-7705bd2a9b9d tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:01.015390 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d2b59dcf-cebb-45af-9016-7705bd2a9b9d tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:01.016202 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d2b59dcf-cebb-45af-9016-7705bd2a9b9d tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:01.016864 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d2b59dcf-cebb-45af-9016-7705bd2a9b9d tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:01.017642 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d2b59dcf-cebb-45af-9016-7705bd2a9b9d tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:01.245992 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-d2b59dcf-cebb-45af-9016-7705bd2a9b9d tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:01.255962 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-d2b59dcf-cebb-45af-9016-7705bd2a9b9d tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d" status: 200 len: 1508 microversion: 2.1 time: 0.254225 Sep 04 01:09:01.256752 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 324/664] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:01 2022] GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d => generated 1508 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:01.284087 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-8568337a-a279-4c92-9146-1f01003f92bc tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "33a19982-0255-4cde-876d-bd618917fddc", "device": "/dev/sdb"}} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:09:01.290153 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-8568337a-a279-4c92-9146-1f01003f92bc tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: a3994b76-60e7-4d9a-a655-2fb09eb68a5d] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:01.297523 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8568337a-a279-4c92-9146-1f01003f92bc tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:01.298393 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8568337a-a279-4c92-9146-1f01003f92bc tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:01.299156 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8568337a-a279-4c92-9146-1f01003f92bc tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:01.300222 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8568337a-a279-4c92-9146-1f01003f92bc tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:01.300922 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8568337a-a279-4c92-9146-1f01003f92bc tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:01.301687 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8568337a-a279-4c92-9146-1f01003f92bc tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:01.486440 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-3559cace-b70c-4f66-a5cf-7fde6ddb27e0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:01.487051 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-3559cace-b70c-4f66-a5cf-7fde6ddb27e0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:01.493775 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3559cace-b70c-4f66-a5cf-7fde6ddb27e0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:01.494197 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3559cace-b70c-4f66-a5cf-7fde6ddb27e0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:01.494634 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3559cace-b70c-4f66-a5cf-7fde6ddb27e0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:01.495920 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3559cace-b70c-4f66-a5cf-7fde6ddb27e0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:01.496378 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3559cace-b70c-4f66-a5cf-7fde6ddb27e0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:01.496805 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3559cace-b70c-4f66-a5cf-7fde6ddb27e0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:01.655952 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-3559cace-b70c-4f66-a5cf-7fde6ddb27e0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:01.665756 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-3559cace-b70c-4f66-a5cf-7fde6ddb27e0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1735 microversion: 2.1 time: 0.183350 Sep 04 01:09:01.666325 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 325/665] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:01 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1735 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:09:01.960264 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-d5a1b0cc-d6bf-45b3-ae70-d54e62869337 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:01.962123 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-d5a1b0cc-d6bf-45b3-ae70-d54e62869337 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:01.969854 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d5a1b0cc-d6bf-45b3-ae70-d54e62869337 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:01.970401 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d5a1b0cc-d6bf-45b3-ae70-d54e62869337 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:01.970841 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d5a1b0cc-d6bf-45b3-ae70-d54e62869337 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:01.971732 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d5a1b0cc-d6bf-45b3-ae70-d54e62869337 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:01.972201 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d5a1b0cc-d6bf-45b3-ae70-d54e62869337 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:01.974049 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d5a1b0cc-d6bf-45b3-ae70-d54e62869337 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:02.021372 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-8568337a-a279-4c92-9146-1f01003f92bc tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "POST /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.742244 Sep 04 01:09:02.022000 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 341/666] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Sep 4 01:09:01 2022] POST /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d/os-volume_attachments => generated 194 bytes in 744 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:09:02.189213 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-d5a1b0cc-d6bf-45b3-ae70-d54e62869337 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:02.198039 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-d5a1b0cc-d6bf-45b3-ae70-d54e62869337 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1674 microversion: 2.1 time: 0.244163 Sep 04 01:09:02.201710 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 326/667] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:01 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1674 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:09:02.692344 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-e3bbc9a7-625a-497c-b05a-7f3dd07986eb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:02.692940 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-e3bbc9a7-625a-497c-b05a-7f3dd07986eb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:02.699670 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e3bbc9a7-625a-497c-b05a-7f3dd07986eb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:02.700180 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e3bbc9a7-625a-497c-b05a-7f3dd07986eb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:02.700608 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e3bbc9a7-625a-497c-b05a-7f3dd07986eb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:02.701321 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e3bbc9a7-625a-497c-b05a-7f3dd07986eb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:02.701765 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e3bbc9a7-625a-497c-b05a-7f3dd07986eb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:02.702182 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e3bbc9a7-625a-497c-b05a-7f3dd07986eb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:02.866537 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e3bbc9a7-625a-497c-b05a-7f3dd07986eb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:02.875484 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-e3bbc9a7-625a-497c-b05a-7f3dd07986eb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1735 microversion: 2.1 time: 0.187403 Sep 04 01:09:02.876472 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 342/668] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:02 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1735 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:03.224494 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-ac0bdf4c-2fd4-41da-9795-99b5396e30d1 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:03.225917 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-ac0bdf4c-2fd4-41da-9795-99b5396e30d1 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:03.234012 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ac0bdf4c-2fd4-41da-9795-99b5396e30d1 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:03.235524 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ac0bdf4c-2fd4-41da-9795-99b5396e30d1 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:03.236149 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ac0bdf4c-2fd4-41da-9795-99b5396e30d1 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:03.237023 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ac0bdf4c-2fd4-41da-9795-99b5396e30d1 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:03.237851 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ac0bdf4c-2fd4-41da-9795-99b5396e30d1 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:03.238448 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ac0bdf4c-2fd4-41da-9795-99b5396e30d1 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:03.397353 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-ac0bdf4c-2fd4-41da-9795-99b5396e30d1 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:03.405990 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-ac0bdf4c-2fd4-41da-9795-99b5396e30d1 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1674 microversion: 2.1 time: 0.185925 Sep 04 01:09:03.406795 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 327/669] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:03 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1674 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:03.900753 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-40ef3013-869f-4704-8afb-fc775790929b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:03.901398 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-40ef3013-869f-4704-8afb-fc775790929b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:03.907963 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-40ef3013-869f-4704-8afb-fc775790929b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:03.908437 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-40ef3013-869f-4704-8afb-fc775790929b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:03.908871 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-40ef3013-869f-4704-8afb-fc775790929b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:03.909600 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-40ef3013-869f-4704-8afb-fc775790929b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:03.910044 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-40ef3013-869f-4704-8afb-fc775790929b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:03.910467 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-40ef3013-869f-4704-8afb-fc775790929b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:04.046333 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-40ef3013-869f-4704-8afb-fc775790929b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:04.057025 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-40ef3013-869f-4704-8afb-fc775790929b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1735 microversion: 2.1 time: 0.160099 Sep 04 01:09:04.058036 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 343/670] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:03 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1735 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:04.433901 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-fb607236-6a25-4d23-a757-2b3554bf88f5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:04.434725 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-fb607236-6a25-4d23-a757-2b3554bf88f5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:04.441578 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fb607236-6a25-4d23-a757-2b3554bf88f5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:04.442117 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fb607236-6a25-4d23-a757-2b3554bf88f5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:04.442583 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fb607236-6a25-4d23-a757-2b3554bf88f5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:04.443471 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fb607236-6a25-4d23-a757-2b3554bf88f5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:04.443925 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fb607236-6a25-4d23-a757-2b3554bf88f5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:04.444741 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fb607236-6a25-4d23-a757-2b3554bf88f5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:04.587840 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-fb607236-6a25-4d23-a757-2b3554bf88f5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:04.596569 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-fb607236-6a25-4d23-a757-2b3554bf88f5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1674 microversion: 2.1 time: 0.166395 Sep 04 01:09:04.610278 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 328/671] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:04 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1674 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:05.081663 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-afb46287-b7cb-4af0-9da1-469b77521b0e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:05.087382 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-afb46287-b7cb-4af0-9da1-469b77521b0e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:05.091158 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-afb46287-b7cb-4af0-9da1-469b77521b0e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:05.091748 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-afb46287-b7cb-4af0-9da1-469b77521b0e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:05.092319 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-afb46287-b7cb-4af0-9da1-469b77521b0e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:05.093293 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-afb46287-b7cb-4af0-9da1-469b77521b0e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:05.093981 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-afb46287-b7cb-4af0-9da1-469b77521b0e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:05.094527 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-afb46287-b7cb-4af0-9da1-469b77521b0e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:05.262002 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-afb46287-b7cb-4af0-9da1-469b77521b0e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:05.273301 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-afb46287-b7cb-4af0-9da1-469b77521b0e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1735 microversion: 2.1 time: 0.195852 Sep 04 01:09:05.273892 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 344/672] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:05 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1735 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:09:05.621129 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-04f12368-7c32-43be-a425-0f1179babe86 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:05.621669 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-04f12368-7c32-43be-a425-0f1179babe86 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:05.628385 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-04f12368-7c32-43be-a425-0f1179babe86 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:05.628864 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-04f12368-7c32-43be-a425-0f1179babe86 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:05.629323 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-04f12368-7c32-43be-a425-0f1179babe86 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:05.630659 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-04f12368-7c32-43be-a425-0f1179babe86 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:05.631107 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-04f12368-7c32-43be-a425-0f1179babe86 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:05.631589 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-04f12368-7c32-43be-a425-0f1179babe86 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:05.795009 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-04f12368-7c32-43be-a425-0f1179babe86 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:05.804673 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-04f12368-7c32-43be-a425-0f1179babe86 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1674 microversion: 2.1 time: 0.187443 Sep 04 01:09:05.819279 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 329/673] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:05 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1674 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:06.296442 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-a8f134ba-6027-43e5-add6-eb9344ba827f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:06.297112 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-a8f134ba-6027-43e5-add6-eb9344ba827f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:06.304336 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a8f134ba-6027-43e5-add6-eb9344ba827f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:06.305145 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a8f134ba-6027-43e5-add6-eb9344ba827f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:06.305786 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a8f134ba-6027-43e5-add6-eb9344ba827f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:06.306700 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a8f134ba-6027-43e5-add6-eb9344ba827f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:06.307380 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a8f134ba-6027-43e5-add6-eb9344ba827f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:06.307934 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a8f134ba-6027-43e5-add6-eb9344ba827f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:06.491234 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-a8f134ba-6027-43e5-add6-eb9344ba827f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:06.502777 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-a8f134ba-6027-43e5-add6-eb9344ba827f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1735 microversion: 2.1 time: 0.209883 Sep 04 01:09:06.505223 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 345/674] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:06 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1735 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:06.828881 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-c0b7010b-219a-4575-a80a-167177a26933 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:06.829509 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-c0b7010b-219a-4575-a80a-167177a26933 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:06.837341 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c0b7010b-219a-4575-a80a-167177a26933 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:06.837837 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c0b7010b-219a-4575-a80a-167177a26933 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:06.838507 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c0b7010b-219a-4575-a80a-167177a26933 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:06.839214 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c0b7010b-219a-4575-a80a-167177a26933 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:06.839739 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c0b7010b-219a-4575-a80a-167177a26933 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:06.840365 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c0b7010b-219a-4575-a80a-167177a26933 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:06.970338 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-c0b7010b-219a-4575-a80a-167177a26933 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:06.978424 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-c0b7010b-219a-4575-a80a-167177a26933 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1674 microversion: 2.1 time: 0.153488 Sep 04 01:09:06.978992 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 330/675] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:06 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1674 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:07.534778 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-9bedb6a3-94ff-421b-9625-eae9b5b19232 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:07.537039 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-9bedb6a3-94ff-421b-9625-eae9b5b19232 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:07.542709 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9bedb6a3-94ff-421b-9625-eae9b5b19232 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:07.543188 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9bedb6a3-94ff-421b-9625-eae9b5b19232 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:07.543761 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9bedb6a3-94ff-421b-9625-eae9b5b19232 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:07.544476 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9bedb6a3-94ff-421b-9625-eae9b5b19232 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:07.544925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9bedb6a3-94ff-421b-9625-eae9b5b19232 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:07.545375 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9bedb6a3-94ff-421b-9625-eae9b5b19232 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:07.689355 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-9bedb6a3-94ff-421b-9625-eae9b5b19232 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:07.699141 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-9bedb6a3-94ff-421b-9625-eae9b5b19232 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1735 microversion: 2.1 time: 0.168323 Sep 04 01:09:07.703362 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 346/676] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:07 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1735 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:08.006463 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-84362ce4-b662-4aef-8c02-a04fd9857363 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:08.007042 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-84362ce4-b662-4aef-8c02-a04fd9857363 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:08.024016 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-84362ce4-b662-4aef-8c02-a04fd9857363 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:08.034576 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-84362ce4-b662-4aef-8c02-a04fd9857363 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:08.035177 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-84362ce4-b662-4aef-8c02-a04fd9857363 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:08.036871 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-84362ce4-b662-4aef-8c02-a04fd9857363 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:08.038156 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-84362ce4-b662-4aef-8c02-a04fd9857363 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:08.038851 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-84362ce4-b662-4aef-8c02-a04fd9857363 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:08.211100 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-84362ce4-b662-4aef-8c02-a04fd9857363 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:08.219478 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-84362ce4-b662-4aef-8c02-a04fd9857363 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1674 microversion: 2.1 time: 0.220102 Sep 04 01:09:08.233394 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 331/677] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:07 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1674 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:08.664259 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-2bb2d891-a928-45f5-9c58-420a7a5e37cb tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-688736299", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-832617863", "security_groups": [{"name": "tempest-secgroup-smoke-2111062138"}], "networks": [{"uuid": "a4020194-c9cd-46d3-9c56-5d8c215c248f"}], "block_device_mapping_v2": [{"uuid": "1e449557-a8b2-47e2-9424-fe3daccd3c83", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:09:08.729964 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-fc4d6066-6a10-428d-8daf-92bc329a0e9d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:08.731036 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-fc4d6066-6a10-428d-8daf-92bc329a0e9d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:08.752744 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fc4d6066-6a10-428d-8daf-92bc329a0e9d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:08.753272 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fc4d6066-6a10-428d-8daf-92bc329a0e9d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:08.753809 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fc4d6066-6a10-428d-8daf-92bc329a0e9d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:08.765177 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fc4d6066-6a10-428d-8daf-92bc329a0e9d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:08.765864 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fc4d6066-6a10-428d-8daf-92bc329a0e9d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:08.766577 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fc4d6066-6a10-428d-8daf-92bc329a0e9d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:08.824127 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.network.neutron [None req-2bb2d891-a928-45f5-9c58-420a7a5e37cb tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] validate_networks() for [('a4020194-c9cd-46d3-9c56-5d8c215c248f', None, None, None, None, None)] {{(pid=121036) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 04 01:09:08.957331 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-fc4d6066-6a10-428d-8daf-92bc329a0e9d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:08.971440 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-fc4d6066-6a10-428d-8daf-92bc329a0e9d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1735 microversion: 2.1 time: 0.251115 Sep 04 01:09:08.972038 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 332/678] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:08 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1735 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:09:09.244521 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-fb833361-a44f-4878-8cbf-aaf11349cfdd tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:09.245368 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-fb833361-a44f-4878-8cbf-aaf11349cfdd tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:09.251949 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fb833361-a44f-4878-8cbf-aaf11349cfdd tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:09.252760 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fb833361-a44f-4878-8cbf-aaf11349cfdd tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:09.253986 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fb833361-a44f-4878-8cbf-aaf11349cfdd tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:09.255061 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fb833361-a44f-4878-8cbf-aaf11349cfdd tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:09.255721 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fb833361-a44f-4878-8cbf-aaf11349cfdd tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:09.257269 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fb833361-a44f-4878-8cbf-aaf11349cfdd tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:09.277823 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=100 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:09:09.281141 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-2bb2d891-a928-45f5-9c58-420a7a5e37cb tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Flavor limits 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 04 01:09:09.281909 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-2bb2d891-a928-45f5-9c58-420a7a5e37cb tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Image limits 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 04 01:09:09.283059 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-2bb2d891-a928-45f5-9c58-420a7a5e37cb tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Flavor pref 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 04 01:09:09.283727 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-2bb2d891-a928-45f5-9c58-420a7a5e37cb tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Image pref 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 04 01:09:09.284299 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-2bb2d891-a928-45f5-9c58-420a7a5e37cb tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 04 01:09:09.297058 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2bb2d891-a928-45f5-9c58-420a7a5e37cb tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:09.309899 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2bb2d891-a928-45f5-9c58-420a7a5e37cb tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:09.310853 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2bb2d891-a928-45f5-9c58-420a7a5e37cb tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:09.311690 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2bb2d891-a928-45f5-9c58-420a7a5e37cb tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:09.312169 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2bb2d891-a928-45f5-9c58-420a7a5e37cb tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:09.312695 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2bb2d891-a928-45f5-9c58-420a7a5e37cb tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:09.340155 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-2bb2d891-a928-45f5-9c58-420a7a5e37cb tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Getting quotas for project 735896cd6b3b4c448dd94b3bf7577cb8. Resources: {'ram', 'instances', 'cores'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 04 01:09:09.347944 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-2bb2d891-a928-45f5-9c58-420a7a5e37cb tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Getting quotas for user 13182cef4cf8458ab34779ffd7514c6c and project 735896cd6b3b4c448dd94b3bf7577cb8. Resources: {'ram', 'instances', 'cores'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 04 01:09:09.428695 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-2bb2d891-a928-45f5-9c58-420a7a5e37cb tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Going to run 1 instances... {{(pid=121036) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 04 01:09:09.432217 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-2bb2d891-a928-45f5-9c58-420a7a5e37cb tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='1e449557-a8b2-47e2-9424-fe3daccd3c83',volume_size=None,volume_type=None)] {{(pid=121036) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 04 01:09:09.435541 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-fb833361-a44f-4878-8cbf-aaf11349cfdd tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:09.445157 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-fb833361-a44f-4878-8cbf-aaf11349cfdd tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1674 microversion: 2.1 time: 0.204153 Sep 04 01:09:09.446429 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 333/679] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:09 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1674 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:09.950363 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-2bb2d891-a928-45f5-9c58-420a7a5e37cb tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.291559 Sep 04 01:09:09.958193 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 347/680] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 4 01:09:08 2022] POST /compute/v2.1/servers => generated 406 bytes in 1300 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 04 01:09:09.964257 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-b98f5290-0cd9-412b-9155-727670b7ecd4 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:09.964898 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-b98f5290-0cd9-412b-9155-727670b7ecd4 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:10.011173 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-80db5408-aefc-4873-9fe6-f1fcefc4e648 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:10.012714 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-80db5408-aefc-4873-9fe6-f1fcefc4e648 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:10.021745 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-80db5408-aefc-4873-9fe6-f1fcefc4e648 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:10.022724 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-80db5408-aefc-4873-9fe6-f1fcefc4e648 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:10.023405 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-80db5408-aefc-4873-9fe6-f1fcefc4e648 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:10.024603 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-80db5408-aefc-4873-9fe6-f1fcefc4e648 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:10.025309 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-80db5408-aefc-4873-9fe6-f1fcefc4e648 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:10.027000 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-80db5408-aefc-4873-9fe6-f1fcefc4e648 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:10.077779 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-b98f5290-0cd9-412b-9155-727670b7ecd4 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:10.086087 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-b98f5290-0cd9-412b-9155-727670b7ecd4 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1087 microversion: 2.1 time: 0.126403 Sep 04 01:09:10.092065 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 334/681] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:09 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1087 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:10.232877 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-80db5408-aefc-4873-9fe6-f1fcefc4e648 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:10.248187 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-80db5408-aefc-4873-9fe6-f1fcefc4e648 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1735 microversion: 2.1 time: 0.249267 Sep 04 01:09:10.260340 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 348/682] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:09 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1735 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:10.470692 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-1e0c80a3-ab9b-4510-a42a-df5fd36be62a tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:10.473383 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-1e0c80a3-ab9b-4510-a42a-df5fd36be62a tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:10.478282 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1e0c80a3-ab9b-4510-a42a-df5fd36be62a tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:10.478763 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1e0c80a3-ab9b-4510-a42a-df5fd36be62a tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:10.479391 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1e0c80a3-ab9b-4510-a42a-df5fd36be62a tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:10.480091 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1e0c80a3-ab9b-4510-a42a-df5fd36be62a tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:10.480621 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1e0c80a3-ab9b-4510-a42a-df5fd36be62a tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:10.481022 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1e0c80a3-ab9b-4510-a42a-df5fd36be62a tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:10.645033 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-1e0c80a3-ab9b-4510-a42a-df5fd36be62a tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:10.657697 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-1e0c80a3-ab9b-4510-a42a-df5fd36be62a tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1674 microversion: 2.1 time: 0.191311 Sep 04 01:09:10.658315 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 335/683] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:10 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1674 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:11.127604 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-e05b7b78-f02e-433d-81e4-a60eed710afc tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:11.128351 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-e05b7b78-f02e-433d-81e4-a60eed710afc tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:11.212366 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e05b7b78-f02e-433d-81e4-a60eed710afc tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:11.220478 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-e05b7b78-f02e-433d-81e4-a60eed710afc tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1087 microversion: 2.1 time: 0.098126 Sep 04 01:09:11.225107 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 349/684] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:11 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1087 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:11.274912 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-2db163d0-e0bd-4a74-b76e-d0016834e59f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:11.276032 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-2db163d0-e0bd-4a74-b76e-d0016834e59f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:11.290818 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2db163d0-e0bd-4a74-b76e-d0016834e59f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:11.291308 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2db163d0-e0bd-4a74-b76e-d0016834e59f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:11.291770 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2db163d0-e0bd-4a74-b76e-d0016834e59f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:11.293313 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2db163d0-e0bd-4a74-b76e-d0016834e59f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:11.295196 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2db163d0-e0bd-4a74-b76e-d0016834e59f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:11.296626 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2db163d0-e0bd-4a74-b76e-d0016834e59f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:11.474405 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-2db163d0-e0bd-4a74-b76e-d0016834e59f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:11.485013 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-2db163d0-e0bd-4a74-b76e-d0016834e59f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1735 microversion: 2.1 time: 0.215484 Sep 04 01:09:11.486104 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 336/685] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:11 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1735 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:11.688000 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-a96e9254-dc87-499d-86d6-d33ba6cc2739 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:11.688928 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-a96e9254-dc87-499d-86d6-d33ba6cc2739 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:11.695793 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a96e9254-dc87-499d-86d6-d33ba6cc2739 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:11.696547 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a96e9254-dc87-499d-86d6-d33ba6cc2739 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:11.696989 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a96e9254-dc87-499d-86d6-d33ba6cc2739 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:11.697784 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a96e9254-dc87-499d-86d6-d33ba6cc2739 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:11.698770 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a96e9254-dc87-499d-86d6-d33ba6cc2739 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:11.699196 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a96e9254-dc87-499d-86d6-d33ba6cc2739 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:11.846536 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-a96e9254-dc87-499d-86d6-d33ba6cc2739 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:11.855484 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-a96e9254-dc87-499d-86d6-d33ba6cc2739 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1674 microversion: 2.1 time: 0.171883 Sep 04 01:09:11.856280 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 350/686] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:11 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1674 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:12.246272 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-54bbd7ae-a38a-4ca9-a964-57c013e014a0 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:12.246882 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-54bbd7ae-a38a-4ca9-a964-57c013e014a0 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:12.253576 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-54bbd7ae-a38a-4ca9-a964-57c013e014a0 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:12.254320 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-54bbd7ae-a38a-4ca9-a964-57c013e014a0 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:12.255781 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-54bbd7ae-a38a-4ca9-a964-57c013e014a0 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:12.256709 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-54bbd7ae-a38a-4ca9-a964-57c013e014a0 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:12.257357 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-54bbd7ae-a38a-4ca9-a964-57c013e014a0 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:12.257951 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-54bbd7ae-a38a-4ca9-a964-57c013e014a0 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:12.331307 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-54bbd7ae-a38a-4ca9-a964-57c013e014a0 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:12.340681 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-54bbd7ae-a38a-4ca9-a964-57c013e014a0 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1129 microversion: 2.1 time: 0.098563 Sep 04 01:09:12.341309 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 337/687] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:12 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1129 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:09:12.513934 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-81b39860-d554-48db-b955-d7110a7afba5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:12.515304 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-81b39860-d554-48db-b955-d7110a7afba5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:12.522790 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-81b39860-d554-48db-b955-d7110a7afba5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:12.523405 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-81b39860-d554-48db-b955-d7110a7afba5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:12.524111 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-81b39860-d554-48db-b955-d7110a7afba5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:12.524944 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-81b39860-d554-48db-b955-d7110a7afba5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:12.525508 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-81b39860-d554-48db-b955-d7110a7afba5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:12.526282 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-81b39860-d554-48db-b955-d7110a7afba5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:12.677607 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-81b39860-d554-48db-b955-d7110a7afba5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:12.688360 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-81b39860-d554-48db-b955-d7110a7afba5 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1735 microversion: 2.1 time: 0.178373 Sep 04 01:09:12.689228 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 351/688] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:12 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1735 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:12.881043 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-6ab2a361-5d10-4944-abf0-e7373798c6bb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:12.881674 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-6ab2a361-5d10-4944-abf0-e7373798c6bb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:12.888731 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6ab2a361-5d10-4944-abf0-e7373798c6bb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:12.889252 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6ab2a361-5d10-4944-abf0-e7373798c6bb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:12.889689 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6ab2a361-5d10-4944-abf0-e7373798c6bb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:12.890390 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6ab2a361-5d10-4944-abf0-e7373798c6bb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:12.890841 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6ab2a361-5d10-4944-abf0-e7373798c6bb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:12.891461 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6ab2a361-5d10-4944-abf0-e7373798c6bb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:13.044915 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-6ab2a361-5d10-4944-abf0-e7373798c6bb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:13.053447 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-6ab2a361-5d10-4944-abf0-e7373798c6bb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1674 microversion: 2.1 time: 0.176801 Sep 04 01:09:13.070560 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 338/689] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:12 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1674 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:13.365512 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-5ee8629a-8c93-4e74-abf4-2a947a95af3d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:13.366253 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-5ee8629a-8c93-4e74-abf4-2a947a95af3d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:13.374490 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5ee8629a-8c93-4e74-abf4-2a947a95af3d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:13.375051 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5ee8629a-8c93-4e74-abf4-2a947a95af3d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:13.375513 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5ee8629a-8c93-4e74-abf4-2a947a95af3d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:13.376258 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5ee8629a-8c93-4e74-abf4-2a947a95af3d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:13.376730 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5ee8629a-8c93-4e74-abf4-2a947a95af3d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:13.377230 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5ee8629a-8c93-4e74-abf4-2a947a95af3d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:13.441580 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-5ee8629a-8c93-4e74-abf4-2a947a95af3d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:13.450842 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-5ee8629a-8c93-4e74-abf4-2a947a95af3d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1129 microversion: 2.1 time: 0.090868 Sep 04 01:09:13.451364 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 352/690] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:13 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1129 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:13.661386 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-0bf7121b-75f7-4935-989a-07d4d23fa905 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:13.662188 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-0bf7121b-75f7-4935-989a-07d4d23fa905 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:13.669517 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0bf7121b-75f7-4935-989a-07d4d23fa905 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:13.670113 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0bf7121b-75f7-4935-989a-07d4d23fa905 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:13.670551 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0bf7121b-75f7-4935-989a-07d4d23fa905 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:13.671486 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0bf7121b-75f7-4935-989a-07d4d23fa905 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:13.671975 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0bf7121b-75f7-4935-989a-07d4d23fa905 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:13.672406 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0bf7121b-75f7-4935-989a-07d4d23fa905 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:13.698382 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-0bf7121b-75f7-4935-989a-07d4d23fa905 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59/ips" status: 200 len: 150 microversion: 2.1 time: 0.039757 Sep 04 01:09:13.698382 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 339/691] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Sep 4 01:09:13 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59/ips => generated 150 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:09:13.712222 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-01b9c094-d392-472d-a139-f000891048c2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "c8868d57-3f95-40ba-8bb5-0b47ba862842"}} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:09:13.714672 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-01b9c094-d392-472d-a139-f000891048c2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:13.722163 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-7ff899bf-956e-4db4-9c4e-0f6b243b5235 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:13.722813 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-7ff899bf-956e-4db4-9c4e-0f6b243b5235 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:13.729712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7ff899bf-956e-4db4-9c4e-0f6b243b5235 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:13.730251 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7ff899bf-956e-4db4-9c4e-0f6b243b5235 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:13.730746 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7ff899bf-956e-4db4-9c4e-0f6b243b5235 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:13.731420 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-01b9c094-d392-472d-a139-f000891048c2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:13.731545 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7ff899bf-956e-4db4-9c4e-0f6b243b5235 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:13.731965 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7ff899bf-956e-4db4-9c4e-0f6b243b5235 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:13.732075 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-01b9c094-d392-472d-a139-f000891048c2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:13.732414 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7ff899bf-956e-4db4-9c4e-0f6b243b5235 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:13.732541 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-01b9c094-d392-472d-a139-f000891048c2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:13.733345 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-01b9c094-d392-472d-a139-f000891048c2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:13.733936 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-01b9c094-d392-472d-a139-f000891048c2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:13.734251 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-01b9c094-d392-472d-a139-f000891048c2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:13.831566 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.wsgi [None req-01b9c094-d392-472d-a139-f000891048c2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] HTTP exception thrown: Invalid volume: volume c8868d57-3f95-40ba-8bb5-0b47ba862842 is already attached to instances: 784b2b9b-9890-4f9e-be75-2548b520e138 Sep 04 01:09:13.834132 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-01b9c094-d392-472d-a139-f000891048c2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Request method failure captured: Sep 04 01:09:13.834132 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: request: POST /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59/os-volume_attachments HTTP/1.1 Sep 04 01:09:13.834132 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept: application/json Sep 04 01:09:13.834132 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept-Encoding: identity Sep 04 01:09:13.834132 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Connection: close Sep 04 01:09:13.834132 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Length: 74 Sep 04 01:09:13.834132 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Type: application/json Sep 04 01:09:13.834132 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Host: 10.222.0.42 Sep 04 01:09:13.834132 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Openstack-System-Scope: None Sep 04 01:09:13.834132 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: User-Agent: python-urllib3/1.26.12 Sep 04 01:09:13.834132 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Auth-Token: gAAAAABjE_nLdulKsCh7rP69EyY8StqBiRnQNFlCX5VoWZ9XzP_YTEis0NNuYGIELGjVYA3FFaOCC9PLYuhPddV3AgDhoAQw_S2o9MK0q1A57ehVT3gYS6mSkklOOLScx3a8r0TUFxeOMhMymWFJt03FSLp4K4nMUe8FttsVqGRsky_ik39T3Zg Sep 04 01:09:13.834132 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Id: None Sep 04 01:09:13.834132 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Name: None Sep 04 01:09:13.834132 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Identity-Status: Confirmed Sep 04 01:09:13.834132 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Is-Admin-Project: True Sep 04 01:09:13.834132 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Id: default Sep 04 01:09:13.834132 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Name: Default Sep 04 01:09:13.834132 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Id: 287d111f08ac4b05bd5adec9c40d0620 Sep 04 01:09:13.834132 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Name: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:09:13.834132 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Role: reader,member Sep 04 01:09:13.834132 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Roles: reader,member Sep 04 01:09:13.835538 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_287d111f08ac4b05bd5adec9c40d0620", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:09:13.835538 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:09:13.835538 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Id: 287d111f08ac4b05bd5adec9c40d0620 Sep 04 01:09:13.835538 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:09:13.835538 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User: tempest-AttachVolumeNegativeTest-825474236-project Sep 04 01:09:13.835538 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Id: default Sep 04 01:09:13.835538 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Name: Default Sep 04 01:09:13.835538 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Id: f5e4723258c64ce5885154dded2d3d43 Sep 04 01:09:13.835538 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Name: tempest-AttachVolumeNegativeTest-825474236-project Sep 04 01:09:13.835538 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Sep 04 01:09:13.835538 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: {"volumeAttachment": {"volumeId": "c8868d57-3f95-40ba-8bb5-0b47ba862842"}} Sep 04 01:09:13.835538 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: method: > Sep 04 01:09:13.836406 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: exception: Invalid volume: volume c8868d57-3f95-40ba-8bb5-0b47ba862842 is already attached to instances: 784b2b9b-9890-4f9e-be75-2548b520e138 Sep 04 01:09:13.836406 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: action_args: {'server_id': '205e4f5b-43ef-494a-a6ad-64f7445e6b59', 'body': {'volumeAttachment': {'volumeId': 'c8868d57-3f95-40ba-8bb5-0b47ba862842'}}} Sep 04 01:09:13.836406 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:09:13.836406 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:09:13.836406 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Sep 04 01:09:13.836406 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Sep 04 01:09:13.836406 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 04 01:09:13.836406 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 04 01:09:13.836406 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 04 01:09:13.836406 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 04 01:09:13.836406 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Sep 04 01:09:13.836406 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Sep 04 01:09:13.836406 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Sep 04 01:09:13.836406 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Sep 04 01:09:13.836406 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume c8868d57-3f95-40ba-8bb5-0b47ba862842 is already attached to instances: 784b2b9b-9890-4f9e-be75-2548b520e138 Sep 04 01:09:13.836406 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:09:13.836406 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:09:13.836406 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:09:13.836406 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:09:13.838270 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:09:13.838270 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:09:13.838270 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:09:13.838270 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:09:13.838270 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:09:13.838270 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:09:13.838270 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 04 01:09:13.838270 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 04 01:09:13.838270 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 04 01:09:13.838270 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 04 01:09:13.838270 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 04 01:09:13.838270 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 04 01:09:13.838270 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Sep 04 01:09:13.838270 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 04 01:09:13.838270 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume c8868d57-3f95-40ba-8bb5-0b47ba862842 is already attached to instances: 784b2b9b-9890-4f9e-be75-2548b520e138 Sep 04 01:09:13.838270 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:09:13.838270 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:09:13.838270 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:09:13.838270 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:09:13.838270 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:09:13.840710 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:09:13.840710 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:09:13.840710 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:09:13.840710 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume c8868d57-3f95-40ba-8bb5-0b47ba862842 is already attached to instances: 784b2b9b-9890-4f9e-be75-2548b520e138 Sep 04 01:09:13.840710 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:09:13.840710 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-01b9c094-d392-472d-a139-f000891048c2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Returning 400 to user: Invalid volume: volume c8868d57-3f95-40ba-8bb5-0b47ba862842 is already attached to instances: 784b2b9b-9890-4f9e-be75-2548b520e138 {{(pid=121036) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:09:13.840710 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-01b9c094-d392-472d-a139-f000891048c2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "POST /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.128914 Sep 04 01:09:13.840710 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 353/692] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Sep 4 01:09:13 2022] POST /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59/os-volume_attachments => generated 174 bytes in 130 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 04 01:09:13.853158 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-6950f94a-25ed-44c4-b46f-918cac3d367b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:13.854224 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-6950f94a-25ed-44c4-b46f-918cac3d367b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:13.869117 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6950f94a-25ed-44c4-b46f-918cac3d367b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:13.869889 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6950f94a-25ed-44c4-b46f-918cac3d367b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:13.870379 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6950f94a-25ed-44c4-b46f-918cac3d367b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:13.871134 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6950f94a-25ed-44c4-b46f-918cac3d367b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:13.871598 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6950f94a-25ed-44c4-b46f-918cac3d367b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:13.872035 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6950f94a-25ed-44c4-b46f-918cac3d367b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:13.905419 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-7ff899bf-956e-4db4-9c4e-0f6b243b5235 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:13.911327 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-6950f94a-25ed-44c4-b46f-918cac3d367b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Going to try to terminate instance {{(pid=121036) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 04 01:09:13.915092 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-7ff899bf-956e-4db4-9c4e-0f6b243b5235 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1735 microversion: 2.1 time: 0.196825 Sep 04 01:09:13.915691 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 340/693] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:13 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1735 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:13.949688 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:09:14.079982 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-2cd34027-2b7f-401d-97f7-92a44b563bd8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:14.083876 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-2cd34027-2b7f-401d-97f7-92a44b563bd8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:14.091314 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2cd34027-2b7f-401d-97f7-92a44b563bd8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:14.094733 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2cd34027-2b7f-401d-97f7-92a44b563bd8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:14.094733 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2cd34027-2b7f-401d-97f7-92a44b563bd8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:14.094733 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2cd34027-2b7f-401d-97f7-92a44b563bd8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:14.094733 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2cd34027-2b7f-401d-97f7-92a44b563bd8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:14.095084 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2cd34027-2b7f-401d-97f7-92a44b563bd8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:14.289038 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-6950f94a-25ed-44c4-b46f-918cac3d367b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "DELETE /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 204 len: 0 microversion: 2.1 time: 0.443900 Sep 04 01:09:14.296859 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 354/694] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 4 01:09:13 2022] DELETE /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 04 01:09:14.297066 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-2cd34027-2b7f-401d-97f7-92a44b563bd8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:14.315876 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b3815e4d-835d-4ac9-88c1-310acab66596 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:14.318122 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-b3815e4d-835d-4ac9-88c1-310acab66596 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:14.334476 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b3815e4d-835d-4ac9-88c1-310acab66596 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:14.335274 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b3815e4d-835d-4ac9-88c1-310acab66596 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:14.335746 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-2cd34027-2b7f-401d-97f7-92a44b563bd8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1674 microversion: 2.1 time: 0.259496 Sep 04 01:09:14.336110 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b3815e4d-835d-4ac9-88c1-310acab66596 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:14.336552 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 341/695] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:14 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1674 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:09:14.338404 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b3815e4d-835d-4ac9-88c1-310acab66596 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:14.339861 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b3815e4d-835d-4ac9-88c1-310acab66596 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:14.341200 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b3815e4d-835d-4ac9-88c1-310acab66596 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:14.480611 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-b35378e7-7383-46cf-898c-95a5989ca518 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:14.481542 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-b35378e7-7383-46cf-898c-95a5989ca518 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:14.495511 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b35378e7-7383-46cf-898c-95a5989ca518 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:14.496006 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b35378e7-7383-46cf-898c-95a5989ca518 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:14.496495 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b35378e7-7383-46cf-898c-95a5989ca518 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:14.497231 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b35378e7-7383-46cf-898c-95a5989ca518 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:14.497685 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b35378e7-7383-46cf-898c-95a5989ca518 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:14.498106 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b35378e7-7383-46cf-898c-95a5989ca518 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:14.546988 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-b3815e4d-835d-4ac9-88c1-310acab66596 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:14.558587 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-b3815e4d-835d-4ac9-88c1-310acab66596 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1683 microversion: 2.1 time: 0.259612 Sep 04 01:09:14.559249 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 355/696] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:14 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1683 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:14.582027 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-b35378e7-7383-46cf-898c-95a5989ca518 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:14.602897 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-b35378e7-7383-46cf-898c-95a5989ca518 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1129 microversion: 2.1 time: 0.130222 Sep 04 01:09:14.603649 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 342/697] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:14 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1129 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:14.942824 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-99285128-00f2-4449-a2dc-2e5ad3bad59d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:14.943437 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-99285128-00f2-4449-a2dc-2e5ad3bad59d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:14.952555 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-99285128-00f2-4449-a2dc-2e5ad3bad59d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:14.953478 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-99285128-00f2-4449-a2dc-2e5ad3bad59d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:14.954144 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-99285128-00f2-4449-a2dc-2e5ad3bad59d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:14.954928 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-99285128-00f2-4449-a2dc-2e5ad3bad59d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:14.955460 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-99285128-00f2-4449-a2dc-2e5ad3bad59d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:14.955960 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-99285128-00f2-4449-a2dc-2e5ad3bad59d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:15.106028 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-99285128-00f2-4449-a2dc-2e5ad3bad59d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:15.116936 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-99285128-00f2-4449-a2dc-2e5ad3bad59d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1735 microversion: 2.1 time: 0.178133 Sep 04 01:09:15.123523 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 356/698] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:14 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1735 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:15.371649 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-d668513b-6e57-4b37-9540-fd0c79560841 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:15.372661 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-d668513b-6e57-4b37-9540-fd0c79560841 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:15.375824 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_api:SELECT=82 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:09:15.394829 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d668513b-6e57-4b37-9540-fd0c79560841 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:15.396042 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d668513b-6e57-4b37-9540-fd0c79560841 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:15.396632 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d668513b-6e57-4b37-9540-fd0c79560841 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:15.397596 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d668513b-6e57-4b37-9540-fd0c79560841 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:15.398147 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d668513b-6e57-4b37-9540-fd0c79560841 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:15.398709 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d668513b-6e57-4b37-9540-fd0c79560841 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:15.560210 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-d668513b-6e57-4b37-9540-fd0c79560841 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:15.570236 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-d668513b-6e57-4b37-9540-fd0c79560841 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1651 microversion: 2.1 time: 0.203912 Sep 04 01:09:15.571155 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 343/699] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:15 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1651 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:15.591282 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-e5de641c-06e9-4368-8072-71295b42f5ab tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:15.593959 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-e5de641c-06e9-4368-8072-71295b42f5ab tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:15.600579 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e5de641c-06e9-4368-8072-71295b42f5ab tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:15.601534 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e5de641c-06e9-4368-8072-71295b42f5ab tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:15.602302 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e5de641c-06e9-4368-8072-71295b42f5ab tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:15.603422 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e5de641c-06e9-4368-8072-71295b42f5ab tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:15.604288 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e5de641c-06e9-4368-8072-71295b42f5ab tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:15.604556 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-3605c5b8-0346-435e-88df-2c0bc3e8023f tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:09:15.605448 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e5de641c-06e9-4368-8072-71295b42f5ab tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:15.612841 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3605c5b8-0346-435e-88df-2c0bc3e8023f tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:15.613586 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3605c5b8-0346-435e-88df-2c0bc3e8023f tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:15.616827 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3605c5b8-0346-435e-88df-2c0bc3e8023f tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:15.646742 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-3605c5b8-0346-435e-88df-2c0bc3e8023f tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Going to try to start instance {{(pid=121037) start /opt/stack/nova/nova/compute/api.py:2785}} Sep 04 01:09:15.801271 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e5de641c-06e9-4368-8072-71295b42f5ab tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:15.810044 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-e5de641c-06e9-4368-8072-71295b42f5ab tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1683 microversion: 2.1 time: 0.227364 Sep 04 01:09:15.813344 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 357/700] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:15 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1683 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:15.819032 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-3aa8f3da-2fa9-45db-ba4c-fb8d2b7e4aef tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:15.819974 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-3aa8f3da-2fa9-45db-ba4c-fb8d2b7e4aef tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:15.829770 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3aa8f3da-2fa9-45db-ba4c-fb8d2b7e4aef tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:15.830279 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3aa8f3da-2fa9-45db-ba4c-fb8d2b7e4aef tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:15.830853 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3aa8f3da-2fa9-45db-ba4c-fb8d2b7e4aef tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:15.831571 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3aa8f3da-2fa9-45db-ba4c-fb8d2b7e4aef tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:15.833327 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3aa8f3da-2fa9-45db-ba4c-fb8d2b7e4aef tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:15.833778 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3aa8f3da-2fa9-45db-ba4c-fb8d2b7e4aef tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:15.895750 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-3605c5b8-0346-435e-88df-2c0bc3e8023f tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "POST /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2/action" status: 202 len: 0 microversion: 2.1 time: 0.295626 Sep 04 01:09:15.897248 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 344/701] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Sep 4 01:09:15 2022] POST /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2/action => generated 0 bytes in 300 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 04 01:09:15.907750 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-bba75219-a3cc-48cf-9602-ab67df3fffc7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:15.908714 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-bba75219-a3cc-48cf-9602-ab67df3fffc7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:15.921821 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-3aa8f3da-2fa9-45db-ba4c-fb8d2b7e4aef tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:15.924599 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-bba75219-a3cc-48cf-9602-ab67df3fffc7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:15.924599 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-bba75219-a3cc-48cf-9602-ab67df3fffc7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:15.924599 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-bba75219-a3cc-48cf-9602-ab67df3fffc7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:15.924599 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-bba75219-a3cc-48cf-9602-ab67df3fffc7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:15.925068 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-bba75219-a3cc-48cf-9602-ab67df3fffc7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:15.925649 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-bba75219-a3cc-48cf-9602-ab67df3fffc7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:15.934286 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-3aa8f3da-2fa9-45db-ba4c-fb8d2b7e4aef tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1185 microversion: 2.1 time: 0.119673 Sep 04 01:09:15.935090 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 358/702] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:15 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1185 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:16.097245 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-bba75219-a3cc-48cf-9602-ab67df3fffc7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:16.107108 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-bba75219-a3cc-48cf-9602-ab67df3fffc7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1660 microversion: 2.1 time: 0.207063 Sep 04 01:09:16.114460 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 345/703] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:15 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1660 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:09:16.142048 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-9643295c-ba2d-4e6e-85e2-fbd97ae7adfb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:16.143275 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-9643295c-ba2d-4e6e-85e2-fbd97ae7adfb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:16.151036 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9643295c-ba2d-4e6e-85e2-fbd97ae7adfb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:16.151742 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9643295c-ba2d-4e6e-85e2-fbd97ae7adfb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:16.152386 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9643295c-ba2d-4e6e-85e2-fbd97ae7adfb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:16.153302 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9643295c-ba2d-4e6e-85e2-fbd97ae7adfb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:16.153916 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9643295c-ba2d-4e6e-85e2-fbd97ae7adfb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:16.154510 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9643295c-ba2d-4e6e-85e2-fbd97ae7adfb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:16.302209 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-9643295c-ba2d-4e6e-85e2-fbd97ae7adfb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:16.311470 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-9643295c-ba2d-4e6e-85e2-fbd97ae7adfb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1735 microversion: 2.1 time: 0.173374 Sep 04 01:09:16.312049 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 359/704] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:16 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1735 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:16.835942 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-dc08bbab-5e70-4c78-9665-af3de8d4fc5a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:16.836658 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-dc08bbab-5e70-4c78-9665-af3de8d4fc5a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:16.843548 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-dc08bbab-5e70-4c78-9665-af3de8d4fc5a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:16.844133 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-dc08bbab-5e70-4c78-9665-af3de8d4fc5a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:16.844637 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-dc08bbab-5e70-4c78-9665-af3de8d4fc5a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:16.845433 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-dc08bbab-5e70-4c78-9665-af3de8d4fc5a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:16.846125 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-dc08bbab-5e70-4c78-9665-af3de8d4fc5a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:16.846752 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-dc08bbab-5e70-4c78-9665-af3de8d4fc5a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:16.967983 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-6cf50552-94df-4fea-91bd-cb5b4ca4ef26 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:16.968955 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-6cf50552-94df-4fea-91bd-cb5b4ca4ef26 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:16.977964 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6cf50552-94df-4fea-91bd-cb5b4ca4ef26 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:16.978455 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6cf50552-94df-4fea-91bd-cb5b4ca4ef26 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:16.978890 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6cf50552-94df-4fea-91bd-cb5b4ca4ef26 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:16.979600 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6cf50552-94df-4fea-91bd-cb5b4ca4ef26 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:16.980050 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6cf50552-94df-4fea-91bd-cb5b4ca4ef26 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:16.980470 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6cf50552-94df-4fea-91bd-cb5b4ca4ef26 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:16.996590 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-dc08bbab-5e70-4c78-9665-af3de8d4fc5a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:17.005474 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-dc08bbab-5e70-4c78-9665-af3de8d4fc5a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1683 microversion: 2.1 time: 0.173386 Sep 04 01:09:17.006638 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 346/705] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:16 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1683 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:17.055771 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-6cf50552-94df-4fea-91bd-cb5b4ca4ef26 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:17.064891 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-6cf50552-94df-4fea-91bd-cb5b4ca4ef26 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1185 microversion: 2.1 time: 0.100814 Sep 04 01:09:17.065580 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 360/706] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:16 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1185 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:09:17.131894 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-5dfc0eb1-89b0-4939-bc63-a586392956a9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:17.132596 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-5dfc0eb1-89b0-4939-bc63-a586392956a9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:17.138812 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5dfc0eb1-89b0-4939-bc63-a586392956a9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:17.139638 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5dfc0eb1-89b0-4939-bc63-a586392956a9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:17.140453 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5dfc0eb1-89b0-4939-bc63-a586392956a9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:17.141787 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5dfc0eb1-89b0-4939-bc63-a586392956a9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:17.142555 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5dfc0eb1-89b0-4939-bc63-a586392956a9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:17.143075 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5dfc0eb1-89b0-4939-bc63-a586392956a9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:17.283457 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-5dfc0eb1-89b0-4939-bc63-a586392956a9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:17.292029 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-5dfc0eb1-89b0-4939-bc63-a586392956a9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1660 microversion: 2.1 time: 0.164053 Sep 04 01:09:17.292593 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 347/707] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:17 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1660 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:17.335174 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-a1f53cae-86e0-42ee-9973-1af39f8b5057 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:17.335845 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-a1f53cae-86e0-42ee-9973-1af39f8b5057 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:17.349895 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a1f53cae-86e0-42ee-9973-1af39f8b5057 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:17.349895 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a1f53cae-86e0-42ee-9973-1af39f8b5057 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:17.349895 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a1f53cae-86e0-42ee-9973-1af39f8b5057 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:17.349895 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a1f53cae-86e0-42ee-9973-1af39f8b5057 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:17.351054 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a1f53cae-86e0-42ee-9973-1af39f8b5057 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:17.351054 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a1f53cae-86e0-42ee-9973-1af39f8b5057 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:17.503451 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-a1f53cae-86e0-42ee-9973-1af39f8b5057 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:17.515765 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-a1f53cae-86e0-42ee-9973-1af39f8b5057 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1735 microversion: 2.1 time: 0.184362 Sep 04 01:09:17.516769 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 361/708] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:17 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1735 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:18.029333 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-ad06adb6-1cf2-4a00-ab83-54f53bb730d3 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:18.029878 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-ad06adb6-1cf2-4a00-ab83-54f53bb730d3 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:18.036219 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ad06adb6-1cf2-4a00-ab83-54f53bb730d3 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:18.036692 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ad06adb6-1cf2-4a00-ab83-54f53bb730d3 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:18.037142 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ad06adb6-1cf2-4a00-ab83-54f53bb730d3 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:18.037825 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ad06adb6-1cf2-4a00-ab83-54f53bb730d3 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:18.038268 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ad06adb6-1cf2-4a00-ab83-54f53bb730d3 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:18.038717 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ad06adb6-1cf2-4a00-ab83-54f53bb730d3 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:18.088405 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-3953cba8-f806-4ce4-9def-60d7a6e8ccfa tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:18.089005 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-3953cba8-f806-4ce4-9def-60d7a6e8ccfa tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:18.094980 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3953cba8-f806-4ce4-9def-60d7a6e8ccfa tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:18.095466 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3953cba8-f806-4ce4-9def-60d7a6e8ccfa tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:18.095894 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3953cba8-f806-4ce4-9def-60d7a6e8ccfa tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:18.096574 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3953cba8-f806-4ce4-9def-60d7a6e8ccfa tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:18.097021 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3953cba8-f806-4ce4-9def-60d7a6e8ccfa tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:18.097465 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3953cba8-f806-4ce4-9def-60d7a6e8ccfa tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:18.167188 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-3953cba8-f806-4ce4-9def-60d7a6e8ccfa tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:18.171370 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-ad06adb6-1cf2-4a00-ab83-54f53bb730d3 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:18.175045 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-3953cba8-f806-4ce4-9def-60d7a6e8ccfa tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1185 microversion: 2.1 time: 0.090444 Sep 04 01:09:18.175682 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 362/709] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:18 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1185 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:18.179538 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-ad06adb6-1cf2-4a00-ab83-54f53bb730d3 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1683 microversion: 2.1 time: 0.154124 Sep 04 01:09:18.180091 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 348/710] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:18 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1683 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:18.316058 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-d18716b3-d581-478a-9efb-12b44fbadfce tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:18.316675 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-d18716b3-d581-478a-9efb-12b44fbadfce tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:18.323773 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d18716b3-d581-478a-9efb-12b44fbadfce tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:18.324297 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d18716b3-d581-478a-9efb-12b44fbadfce tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:18.324727 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d18716b3-d581-478a-9efb-12b44fbadfce tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:18.325382 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d18716b3-d581-478a-9efb-12b44fbadfce tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:18.325826 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d18716b3-d581-478a-9efb-12b44fbadfce tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:18.326246 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d18716b3-d581-478a-9efb-12b44fbadfce tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:18.454271 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-d18716b3-d581-478a-9efb-12b44fbadfce tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:18.462002 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-d18716b3-d581-478a-9efb-12b44fbadfce tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1660 microversion: 2.1 time: 0.149733 Sep 04 01:09:18.462577 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 363/711] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:18 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1660 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:18.540191 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-d56a7620-b2fe-4abb-95b3-c5600f71f2fd tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:18.541010 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-d56a7620-b2fe-4abb-95b3-c5600f71f2fd tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:18.546955 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d56a7620-b2fe-4abb-95b3-c5600f71f2fd tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:18.547440 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d56a7620-b2fe-4abb-95b3-c5600f71f2fd tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:18.547879 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d56a7620-b2fe-4abb-95b3-c5600f71f2fd tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:18.548654 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d56a7620-b2fe-4abb-95b3-c5600f71f2fd tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:18.549125 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d56a7620-b2fe-4abb-95b3-c5600f71f2fd tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:18.549548 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d56a7620-b2fe-4abb-95b3-c5600f71f2fd tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:18.679363 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-d56a7620-b2fe-4abb-95b3-c5600f71f2fd tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:18.687362 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-d56a7620-b2fe-4abb-95b3-c5600f71f2fd tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1735 microversion: 2.1 time: 0.151528 Sep 04 01:09:18.687926 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 349/712] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:18 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1735 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:19.199035 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-061a3011-2c39-4844-bee3-b9cddabc837f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:19.199645 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-061a3011-2c39-4844-bee3-b9cddabc837f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:19.203770 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-4048a366-7173-4aa3-8e48-996a1834af6f tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:19.204388 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-4048a366-7173-4aa3-8e48-996a1834af6f tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:19.206165 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-061a3011-2c39-4844-bee3-b9cddabc837f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:19.208284 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-061a3011-2c39-4844-bee3-b9cddabc837f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:19.208284 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-061a3011-2c39-4844-bee3-b9cddabc837f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:19.208284 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-061a3011-2c39-4844-bee3-b9cddabc837f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:19.208742 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-061a3011-2c39-4844-bee3-b9cddabc837f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:19.209139 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-061a3011-2c39-4844-bee3-b9cddabc837f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:19.213532 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4048a366-7173-4aa3-8e48-996a1834af6f tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:19.214020 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4048a366-7173-4aa3-8e48-996a1834af6f tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:19.214461 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4048a366-7173-4aa3-8e48-996a1834af6f tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:19.215173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4048a366-7173-4aa3-8e48-996a1834af6f tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:19.215619 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4048a366-7173-4aa3-8e48-996a1834af6f tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:19.216056 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4048a366-7173-4aa3-8e48-996a1834af6f tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:19.240242 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:09:19.277266 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-061a3011-2c39-4844-bee3-b9cddabc837f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:19.285387 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-061a3011-2c39-4844-bee3-b9cddabc837f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1185 microversion: 2.1 time: 0.090397 Sep 04 01:09:19.286906 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 364/713] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:19 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1185 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:19.444984 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-4048a366-7173-4aa3-8e48-996a1834af6f tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:19.453598 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-4048a366-7173-4aa3-8e48-996a1834af6f tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1683 microversion: 2.1 time: 0.254065 Sep 04 01:09:19.454182 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 350/714] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:19 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1683 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:09:19.485276 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b4e22c04-b95d-494e-a6e1-d7439b480255 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:19.493306 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-b4e22c04-b95d-494e-a6e1-d7439b480255 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:19.493306 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell0:SELECT=2 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:09:19.493306 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b4e22c04-b95d-494e-a6e1-d7439b480255 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:19.493731 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b4e22c04-b95d-494e-a6e1-d7439b480255 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:19.494167 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b4e22c04-b95d-494e-a6e1-d7439b480255 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:19.494853 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b4e22c04-b95d-494e-a6e1-d7439b480255 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:19.495307 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b4e22c04-b95d-494e-a6e1-d7439b480255 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:19.495731 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b4e22c04-b95d-494e-a6e1-d7439b480255 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:19.653817 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-b4e22c04-b95d-494e-a6e1-d7439b480255 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:19.661713 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-b4e22c04-b95d-494e-a6e1-d7439b480255 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1660 microversion: 2.1 time: 0.180172 Sep 04 01:09:19.662310 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 365/715] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:19 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1660 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:19.710405 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-6b822dd1-b457-471e-a539-bb382ed264a1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:19.711012 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-6b822dd1-b457-471e-a539-bb382ed264a1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:19.717543 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6b822dd1-b457-471e-a539-bb382ed264a1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:19.718018 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6b822dd1-b457-471e-a539-bb382ed264a1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:19.718449 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6b822dd1-b457-471e-a539-bb382ed264a1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:19.719233 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6b822dd1-b457-471e-a539-bb382ed264a1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:19.719682 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6b822dd1-b457-471e-a539-bb382ed264a1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:19.720099 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6b822dd1-b457-471e-a539-bb382ed264a1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:19.945380 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-6b822dd1-b457-471e-a539-bb382ed264a1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:19.953997 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-6b822dd1-b457-471e-a539-bb382ed264a1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1735 microversion: 2.1 time: 0.247658 Sep 04 01:09:19.954660 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 351/716] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:19 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1735 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:20.311013 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-a402c797-9f03-4e85-ac5e-80e6a6ad35c7 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:20.311627 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-a402c797-9f03-4e85-ac5e-80e6a6ad35c7 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:20.317605 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a402c797-9f03-4e85-ac5e-80e6a6ad35c7 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:20.318089 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a402c797-9f03-4e85-ac5e-80e6a6ad35c7 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:20.318718 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a402c797-9f03-4e85-ac5e-80e6a6ad35c7 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:20.319521 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a402c797-9f03-4e85-ac5e-80e6a6ad35c7 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:20.319980 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a402c797-9f03-4e85-ac5e-80e6a6ad35c7 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:20.320407 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a402c797-9f03-4e85-ac5e-80e6a6ad35c7 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:20.386077 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-a402c797-9f03-4e85-ac5e-80e6a6ad35c7 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:20.393995 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-a402c797-9f03-4e85-ac5e-80e6a6ad35c7 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1185 microversion: 2.1 time: 0.086984 Sep 04 01:09:20.409277 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 366/717] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:20 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1185 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:20.478342 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-a0223a15-a274-4ae6-b81f-85761555e07e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:20.478959 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-a0223a15-a274-4ae6-b81f-85761555e07e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:20.480815 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell0:SELECT=2 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:09:20.486462 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a0223a15-a274-4ae6-b81f-85761555e07e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:20.486946 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a0223a15-a274-4ae6-b81f-85761555e07e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:20.487377 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a0223a15-a274-4ae6-b81f-85761555e07e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:20.488069 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a0223a15-a274-4ae6-b81f-85761555e07e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:20.488520 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a0223a15-a274-4ae6-b81f-85761555e07e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:20.488942 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a0223a15-a274-4ae6-b81f-85761555e07e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:20.686474 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-f2d0f157-25ee-433e-8e57-1656831462ce tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:20.687095 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-f2d0f157-25ee-433e-8e57-1656831462ce tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:20.693378 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f2d0f157-25ee-433e-8e57-1656831462ce tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:20.694062 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f2d0f157-25ee-433e-8e57-1656831462ce tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:20.694726 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f2d0f157-25ee-433e-8e57-1656831462ce tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:20.695583 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f2d0f157-25ee-433e-8e57-1656831462ce tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:20.696334 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f2d0f157-25ee-433e-8e57-1656831462ce tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:20.696910 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f2d0f157-25ee-433e-8e57-1656831462ce tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:20.826762 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-a0223a15-a274-4ae6-b81f-85761555e07e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:20.837755 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-a0223a15-a274-4ae6-b81f-85761555e07e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1683 microversion: 2.1 time: 0.363783 Sep 04 01:09:20.840316 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 352/718] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:20 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1683 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:09:20.915162 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-f2d0f157-25ee-433e-8e57-1656831462ce tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:20.922894 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-f2d0f157-25ee-433e-8e57-1656831462ce tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1660 microversion: 2.1 time: 0.240558 Sep 04 01:09:20.923663 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 367/719] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:20 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1660 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:20.979301 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-97f0b1b8-ab40-422d-89b7-4e65a32230d2 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:20.979904 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-97f0b1b8-ab40-422d-89b7-4e65a32230d2 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:20.988213 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-97f0b1b8-ab40-422d-89b7-4e65a32230d2 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:20.988523 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-97f0b1b8-ab40-422d-89b7-4e65a32230d2 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:20.988957 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-97f0b1b8-ab40-422d-89b7-4e65a32230d2 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:20.994948 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-97f0b1b8-ab40-422d-89b7-4e65a32230d2 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:20.995439 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-97f0b1b8-ab40-422d-89b7-4e65a32230d2 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:20.995963 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-97f0b1b8-ab40-422d-89b7-4e65a32230d2 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:21.419964 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-475d8c6c-391d-443a-9b8f-4afc7ab57e0d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:21.421588 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-475d8c6c-391d-443a-9b8f-4afc7ab57e0d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:21.424684 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell0:SELECT=8 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:09:21.440347 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-475d8c6c-391d-443a-9b8f-4afc7ab57e0d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:21.441780 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-475d8c6c-391d-443a-9b8f-4afc7ab57e0d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:21.442643 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-475d8c6c-391d-443a-9b8f-4afc7ab57e0d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:21.453725 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-475d8c6c-391d-443a-9b8f-4afc7ab57e0d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:21.453725 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-475d8c6c-391d-443a-9b8f-4afc7ab57e0d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:21.453725 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-475d8c6c-391d-443a-9b8f-4afc7ab57e0d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:21.531779 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-475d8c6c-391d-443a-9b8f-4afc7ab57e0d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:21.540333 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-475d8c6c-391d-443a-9b8f-4afc7ab57e0d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1203 microversion: 2.1 time: 0.126719 Sep 04 01:09:21.540906 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 368/720] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:21 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1203 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:21.564326 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-97f0b1b8-ab40-422d-89b7-4e65a32230d2 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:21.577231 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-97f0b1b8-ab40-422d-89b7-4e65a32230d2 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1735 microversion: 2.1 time: 0.601435 Sep 04 01:09:21.583002 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 353/721] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:20 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1735 bytes in 608 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:21.868222 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-dac8f521-55e3-41fe-91c9-49b5a0f8051e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:21.868828 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-dac8f521-55e3-41fe-91c9-49b5a0f8051e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:21.879900 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dac8f521-55e3-41fe-91c9-49b5a0f8051e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:21.880394 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dac8f521-55e3-41fe-91c9-49b5a0f8051e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:21.880858 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dac8f521-55e3-41fe-91c9-49b5a0f8051e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:21.881743 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dac8f521-55e3-41fe-91c9-49b5a0f8051e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:21.882362 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dac8f521-55e3-41fe-91c9-49b5a0f8051e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:21.883650 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dac8f521-55e3-41fe-91c9-49b5a0f8051e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:21.949194 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-22a7b140-f2ab-4ed0-85f2-7e4fbe4cac50 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:21.950095 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-22a7b140-f2ab-4ed0-85f2-7e4fbe4cac50 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:21.959897 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-22a7b140-f2ab-4ed0-85f2-7e4fbe4cac50 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:21.960882 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-22a7b140-f2ab-4ed0-85f2-7e4fbe4cac50 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:21.961625 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-22a7b140-f2ab-4ed0-85f2-7e4fbe4cac50 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:21.962377 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-22a7b140-f2ab-4ed0-85f2-7e4fbe4cac50 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:21.963346 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-dac8f521-55e3-41fe-91c9-49b5a0f8051e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:21.973934 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-dac8f521-55e3-41fe-91c9-49b5a0f8051e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1615 microversion: 2.1 time: 0.110044 Sep 04 01:09:21.980156 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 369/722] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:21 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1615 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:21.980339 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-22a7b140-f2ab-4ed0-85f2-7e4fbe4cac50 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:21.980339 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-22a7b140-f2ab-4ed0-85f2-7e4fbe4cac50 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:22.059557 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-218182ae-2438-4823-a264-b150bc3c96b0 req-522240c9-55d9-495d-9ee6-f9275af8c66b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "205e4f5b-43ef-494a-a6ad-64f7445e6b59", "name": "network-vif-deleted", "tag": "610a3f9e-176f-46b6-82d0-90b3e3651b7b"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:09:22.082798 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-218182ae-2438-4823-a264-b150bc3c96b0 req-522240c9-55d9-495d-9ee6-f9275af8c66b service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:22.083306 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-218182ae-2438-4823-a264-b150bc3c96b0 req-522240c9-55d9-495d-9ee6-f9275af8c66b service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:22.083732 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-218182ae-2438-4823-a264-b150bc3c96b0 req-522240c9-55d9-495d-9ee6-f9275af8c66b service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:22.113447 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-218182ae-2438-4823-a264-b150bc3c96b0 req-522240c9-55d9-495d-9ee6-f9275af8c66b service nova] Creating event network-vif-deleted:610a3f9e-176f-46b6-82d0-90b3e3651b7b for instance 205e4f5b-43ef-494a-a6ad-64f7445e6b59 on hv-cinder-maste Sep 04 01:09:22.126953 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-218182ae-2438-4823-a264-b150bc3c96b0 req-522240c9-55d9-495d-9ee6-f9275af8c66b service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.071598 Sep 04 01:09:22.127572 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 370/723] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:09:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:09:22.250954 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-22a7b140-f2ab-4ed0-85f2-7e4fbe4cac50 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:22.261527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-22a7b140-f2ab-4ed0-85f2-7e4fbe4cac50 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1660 microversion: 2.1 time: 0.318782 Sep 04 01:09:22.277730 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 354/724] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:21 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1660 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:09:22.580110 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-0b16e827-fbc2-46b7-83a9-57984e03071a tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:22.580808 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-0b16e827-fbc2-46b7-83a9-57984e03071a tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:22.594441 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0b16e827-fbc2-46b7-83a9-57984e03071a tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:22.598609 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0b16e827-fbc2-46b7-83a9-57984e03071a tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:22.599246 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0b16e827-fbc2-46b7-83a9-57984e03071a tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:22.602566 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0b16e827-fbc2-46b7-83a9-57984e03071a tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:22.604486 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0b16e827-fbc2-46b7-83a9-57984e03071a tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:22.605837 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-1573f5db-36cf-46aa-b61a-9c988d922d72 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:22.606579 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-1573f5db-36cf-46aa-b61a-9c988d922d72 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:22.607245 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0b16e827-fbc2-46b7-83a9-57984e03071a tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:22.626096 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1573f5db-36cf-46aa-b61a-9c988d922d72 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:22.627310 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1573f5db-36cf-46aa-b61a-9c988d922d72 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:22.629627 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1573f5db-36cf-46aa-b61a-9c988d922d72 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:22.630595 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1573f5db-36cf-46aa-b61a-9c988d922d72 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:22.633513 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1573f5db-36cf-46aa-b61a-9c988d922d72 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:22.634240 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1573f5db-36cf-46aa-b61a-9c988d922d72 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:22.685359 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:09:22.883487 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-0b16e827-fbc2-46b7-83a9-57984e03071a tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:22.894150 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-0b16e827-fbc2-46b7-83a9-57984e03071a tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1271 microversion: 2.1 time: 0.321910 Sep 04 01:09:22.894817 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-1573f5db-36cf-46aa-b61a-9c988d922d72 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:22.896611 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 371/725] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:22 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1271 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:22.916514 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-1573f5db-36cf-46aa-b61a-9c988d922d72 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1735 microversion: 2.1 time: 0.314962 Sep 04 01:09:22.917117 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 355/726] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:22 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1735 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:23.004508 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-6a65a298-b119-4fb9-97d4-d8e0979c8599 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:23.005426 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-6a65a298-b119-4fb9-97d4-d8e0979c8599 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:23.028079 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6a65a298-b119-4fb9-97d4-d8e0979c8599 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:23.033295 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6a65a298-b119-4fb9-97d4-d8e0979c8599 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:23.033809 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6a65a298-b119-4fb9-97d4-d8e0979c8599 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:23.034535 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6a65a298-b119-4fb9-97d4-d8e0979c8599 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:23.035015 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6a65a298-b119-4fb9-97d4-d8e0979c8599 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:23.035452 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6a65a298-b119-4fb9-97d4-d8e0979c8599 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:23.107435 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6a65a298-b119-4fb9-97d4-d8e0979c8599 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:23.107923 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6a65a298-b119-4fb9-97d4-d8e0979c8599 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:23.108353 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6a65a298-b119-4fb9-97d4-d8e0979c8599 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:23.108843 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.objects.instance [None req-6a65a298-b119-4fb9-97d4-d8e0979c8599 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lazy-loading 'fault' on Instance uuid 205e4f5b-43ef-494a-a6ad-64f7445e6b59 {{(pid=121036) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:09:23.182395 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-6a65a298-b119-4fb9-97d4-d8e0979c8599 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:23.191360 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-6a65a298-b119-4fb9-97d4-d8e0979c8599 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 200 len: 1332 microversion: 2.1 time: 0.191151 Sep 04 01:09:23.199134 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 372/727] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:22 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 1332 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:23.287687 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-f7f46cb5-61cc-4649-9e8c-352a4aec7cb9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:23.288548 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-f7f46cb5-61cc-4649-9e8c-352a4aec7cb9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:23.295117 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f7f46cb5-61cc-4649-9e8c-352a4aec7cb9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:23.295793 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f7f46cb5-61cc-4649-9e8c-352a4aec7cb9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:23.296388 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f7f46cb5-61cc-4649-9e8c-352a4aec7cb9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:23.297472 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f7f46cb5-61cc-4649-9e8c-352a4aec7cb9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:23.298080 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f7f46cb5-61cc-4649-9e8c-352a4aec7cb9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:23.298962 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f7f46cb5-61cc-4649-9e8c-352a4aec7cb9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:23.508289 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-09d06a88-3945-41cb-aaa0-c441d59f112a tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:23.508925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-09d06a88-3945-41cb-aaa0-c441d59f112a tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: a3994b76-60e7-4d9a-a655-2fb09eb68a5d] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:23.520537 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-f7f46cb5-61cc-4649-9e8c-352a4aec7cb9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:23.523885 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-09d06a88-3945-41cb-aaa0-c441d59f112a tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:23.524545 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-09d06a88-3945-41cb-aaa0-c441d59f112a tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:23.525114 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-09d06a88-3945-41cb-aaa0-c441d59f112a tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:23.526599 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-09d06a88-3945-41cb-aaa0-c441d59f112a tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:23.529912 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-09d06a88-3945-41cb-aaa0-c441d59f112a tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:23.530606 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-09d06a88-3945-41cb-aaa0-c441d59f112a tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:23.532678 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-f7f46cb5-61cc-4649-9e8c-352a4aec7cb9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1664 microversion: 2.1 time: 0.249465 Sep 04 01:09:23.535818 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 356/728] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:23 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1664 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:23.681964 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.objects.instance [None req-09d06a88-3945-41cb-aaa0-c441d59f112a tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lazy-loading 'services' on Instance uuid a3994b76-60e7-4d9a-a655-2fb09eb68a5d {{(pid=121036) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:09:23.920184 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-47c40fb7-9d44-4cd5-8135-88b2fc6f5ee3 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:23.921422 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-47c40fb7-9d44-4cd5-8135-88b2fc6f5ee3 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:23.931677 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-47c40fb7-9d44-4cd5-8135-88b2fc6f5ee3 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:23.933152 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-47c40fb7-9d44-4cd5-8135-88b2fc6f5ee3 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:23.933946 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-47c40fb7-9d44-4cd5-8135-88b2fc6f5ee3 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:23.935535 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-47c40fb7-9d44-4cd5-8135-88b2fc6f5ee3 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:23.936338 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-47c40fb7-9d44-4cd5-8135-88b2fc6f5ee3 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:23.938614 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-47c40fb7-9d44-4cd5-8135-88b2fc6f5ee3 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:23.941792 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-09d06a88-3945-41cb-aaa0-c441d59f112a tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "DELETE /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d/os-volume_attachments/33a19982-0255-4cde-876d-bd618917fddc" status: 202 len: 0 microversion: 2.1 time: 0.437869 Sep 04 01:09:23.955106 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 373/729] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Sep 4 01:09:23 2022] DELETE /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d/os-volume_attachments/33a19982-0255-4cde-876d-bd618917fddc => generated 0 bytes in 447 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 04 01:09:23.957336 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-5b2e9417-96e1-4d9e-afb6-14b184f7dc75 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:23.957336 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-5b2e9417-96e1-4d9e-afb6-14b184f7dc75 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:23.980720 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5b2e9417-96e1-4d9e-afb6-14b184f7dc75 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:23.984213 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5b2e9417-96e1-4d9e-afb6-14b184f7dc75 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:23.984670 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5b2e9417-96e1-4d9e-afb6-14b184f7dc75 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:23.988585 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5b2e9417-96e1-4d9e-afb6-14b184f7dc75 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:23.988749 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5b2e9417-96e1-4d9e-afb6-14b184f7dc75 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:23.989550 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5b2e9417-96e1-4d9e-afb6-14b184f7dc75 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:24.183095 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-5b2e9417-96e1-4d9e-afb6-14b184f7dc75 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:24.193147 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-5b2e9417-96e1-4d9e-afb6-14b184f7dc75 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1735 microversion: 2.1 time: 0.241736 Sep 04 01:09:24.194074 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 374/730] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:23 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1735 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:24.205925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-b22f43ea-c51d-4720-a93a-1576423f2f90 req-fb601e7f-c4aa-4792-a5dd-27520d38b829 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "74df11dc-31bd-4745-83bc-4b3c430d02ff", "tag": "d072bfcf-41bd-430e-800c-1d1a3b17d915"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:09:24.216827 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-b22f43ea-c51d-4720-a93a-1576423f2f90 req-fb601e7f-c4aa-4792-a5dd-27520d38b829 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:24.216827 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-b22f43ea-c51d-4720-a93a-1576423f2f90 req-fb601e7f-c4aa-4792-a5dd-27520d38b829 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:24.217094 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-b22f43ea-c51d-4720-a93a-1576423f2f90 req-fb601e7f-c4aa-4792-a5dd-27520d38b829 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:24.229321 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-b22f43ea-c51d-4720-a93a-1576423f2f90 req-fb601e7f-c4aa-4792-a5dd-27520d38b829 service nova] Creating event network-changed:d072bfcf-41bd-430e-800c-1d1a3b17d915 for instance 74df11dc-31bd-4745-83bc-4b3c430d02ff on hv-cinder-maste Sep 04 01:09:24.234246 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-b22f43ea-c51d-4720-a93a-1576423f2f90 req-fb601e7f-c4aa-4792-a5dd-27520d38b829 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032732 Sep 04 01:09:24.234910 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 375/731] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:09:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:09:24.235389 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-47c40fb7-9d44-4cd5-8135-88b2fc6f5ee3 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:24.240302 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-4dcf5834-170b-4933-a446-7b8cfb0f13fe tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:24.240998 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-4dcf5834-170b-4933-a446-7b8cfb0f13fe tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:24.244075 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:09:24.246487 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-47c40fb7-9d44-4cd5-8135-88b2fc6f5ee3 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1259 microversion: 2.1 time: 0.330373 Sep 04 01:09:24.248349 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 357/732] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:23 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1259 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:09:24.248556 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4dcf5834-170b-4933-a446-7b8cfb0f13fe tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:24.249010 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4dcf5834-170b-4933-a446-7b8cfb0f13fe tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:24.249472 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4dcf5834-170b-4933-a446-7b8cfb0f13fe tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:24.250168 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4dcf5834-170b-4933-a446-7b8cfb0f13fe tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:24.250625 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4dcf5834-170b-4933-a446-7b8cfb0f13fe tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:24.251036 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4dcf5834-170b-4933-a446-7b8cfb0f13fe tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:24.265240 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.wsgi [None req-4dcf5834-170b-4933-a446-7b8cfb0f13fe tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] HTTP exception thrown: Instance 205e4f5b-43ef-494a-a6ad-64f7445e6b59 could not be found. Sep 04 01:09:24.266861 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-4dcf5834-170b-4933-a446-7b8cfb0f13fe tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Request method failure captured: Sep 04 01:09:24.266861 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: request: GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 HTTP/1.1 Sep 04 01:09:24.266861 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept: application/json Sep 04 01:09:24.266861 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept-Encoding: identity Sep 04 01:09:24.266861 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Connection: close Sep 04 01:09:24.266861 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Length: 0 Sep 04 01:09:24.266861 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Type: application/json Sep 04 01:09:24.266861 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Host: 10.222.0.42 Sep 04 01:09:24.266861 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Openstack-System-Scope: None Sep 04 01:09:24.266861 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: User-Agent: python-urllib3/1.26.12 Sep 04 01:09:24.266861 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Auth-Token: gAAAAABjE_nLdulKsCh7rP69EyY8StqBiRnQNFlCX5VoWZ9XzP_YTEis0NNuYGIELGjVYA3FFaOCC9PLYuhPddV3AgDhoAQw_S2o9MK0q1A57ehVT3gYS6mSkklOOLScx3a8r0TUFxeOMhMymWFJt03FSLp4K4nMUe8FttsVqGRsky_ik39T3Zg Sep 04 01:09:24.266861 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Id: None Sep 04 01:09:24.266861 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Name: None Sep 04 01:09:24.266861 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Identity-Status: Confirmed Sep 04 01:09:24.266861 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Is-Admin-Project: True Sep 04 01:09:24.266861 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Id: default Sep 04 01:09:24.266861 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Name: Default Sep 04 01:09:24.266861 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Id: 287d111f08ac4b05bd5adec9c40d0620 Sep 04 01:09:24.266861 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Name: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:09:24.266861 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Role: reader,member Sep 04 01:09:24.266861 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Roles: reader,member Sep 04 01:09:24.268250 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_287d111f08ac4b05bd5adec9c40d0620", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:09:24.268250 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:09:24.268250 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Id: 287d111f08ac4b05bd5adec9c40d0620 Sep 04 01:09:24.268250 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:09:24.268250 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User: tempest-AttachVolumeNegativeTest-825474236-project Sep 04 01:09:24.268250 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Id: default Sep 04 01:09:24.268250 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Name: Default Sep 04 01:09:24.268250 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Id: f5e4723258c64ce5885154dded2d3d43 Sep 04 01:09:24.268250 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Name: tempest-AttachVolumeNegativeTest-825474236-project Sep 04 01:09:24.268250 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: method: > Sep 04 01:09:24.268250 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: exception: Instance 205e4f5b-43ef-494a-a6ad-64f7445e6b59 could not be found. Sep 04 01:09:24.268250 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: action_args: {'id': '205e4f5b-43ef-494a-a6ad-64f7445e6b59'} Sep 04 01:09:24.269158 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:09:24.269158 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:09:24.269158 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:09:24.269158 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:09:24.269158 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:09:24.269158 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:09:24.269158 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:09:24.269158 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:09:24.269158 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:09:24.269158 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:09:24.269158 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 205e4f5b-43ef-494a-a6ad-64f7445e6b59 could not be found. Sep 04 01:09:24.269158 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:09:24.269158 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:09:24.269158 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:09:24.269158 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:09:24.269158 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:09:24.269158 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:09:24.269158 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:09:24.269158 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:09:24.269158 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:09:24.275034 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:09:24.275034 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 04 01:09:24.275034 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 04 01:09:24.275034 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:09:24.275034 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:09:24.275034 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:09:24.275034 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:09:24.275034 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 205e4f5b-43ef-494a-a6ad-64f7445e6b59 could not be found. Sep 04 01:09:24.275034 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:09:24.275034 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:09:24.275034 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:09:24.275034 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:09:24.275034 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:09:24.275034 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:09:24.275034 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:09:24.275034 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:09:24.275034 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 205e4f5b-43ef-494a-a6ad-64f7445e6b59 could not be found. Sep 04 01:09:24.275034 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:09:24.276347 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-4dcf5834-170b-4933-a446-7b8cfb0f13fe tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Returning 404 to user: Instance 205e4f5b-43ef-494a-a6ad-64f7445e6b59 could not be found. {{(pid=121036) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:09:24.276347 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-4dcf5834-170b-4933-a446-7b8cfb0f13fe tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 404 len: 111 microversion: 2.1 time: 0.032224 Sep 04 01:09:24.276347 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 376/733] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:24 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 04 01:09:24.281562 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-30e96396-ba7f-48f1-bb84-ea086e29ccc6 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '.version_select at 0x7fc10c5c2c10>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:24.359168 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-30e96396-ba7f-48f1-bb84-ea086e29ccc6 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1542700883" status: 202 len: 0 microversion: 2.1 time: 0.081352 Sep 04 01:09:24.359829 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 358/734] 10.222.0.42 () {58 vars in 1221 bytes} [Sun Sep 4 01:09:24 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1542700883 => generated 0 bytes in 83 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 04 01:09:25.217929 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-7450ebc2-9ce2-44e8-8183-416e12215f12 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:25.218545 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-7450ebc2-9ce2-44e8-8183-416e12215f12 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:25.226144 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7450ebc2-9ce2-44e8-8183-416e12215f12 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:25.226635 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7450ebc2-9ce2-44e8-8183-416e12215f12 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:25.228687 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7450ebc2-9ce2-44e8-8183-416e12215f12 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:25.228687 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7450ebc2-9ce2-44e8-8183-416e12215f12 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:25.228687 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7450ebc2-9ce2-44e8-8183-416e12215f12 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:25.229133 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7450ebc2-9ce2-44e8-8183-416e12215f12 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:25.273513 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-73cf8a11-81ac-4bb8-9260-f09368a8e739 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:25.275642 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-73cf8a11-81ac-4bb8-9260-f09368a8e739 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:25.283277 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-73cf8a11-81ac-4bb8-9260-f09368a8e739 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:25.283757 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-73cf8a11-81ac-4bb8-9260-f09368a8e739 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:25.286573 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-73cf8a11-81ac-4bb8-9260-f09368a8e739 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:25.286573 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-73cf8a11-81ac-4bb8-9260-f09368a8e739 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:25.286573 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-73cf8a11-81ac-4bb8-9260-f09368a8e739 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:25.286573 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-73cf8a11-81ac-4bb8-9260-f09368a8e739 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:25.395192 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-7450ebc2-9ce2-44e8-8183-416e12215f12 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:25.403749 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-7450ebc2-9ce2-44e8-8183-416e12215f12 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1735 microversion: 2.1 time: 0.190092 Sep 04 01:09:25.404349 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 377/735] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:25 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1735 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:25.499346 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-73cf8a11-81ac-4bb8-9260-f09368a8e739 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:25.509092 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-5e0601dc-545e-47e1-b534-e9249d1de4c8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:25.509591 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-73cf8a11-81ac-4bb8-9260-f09368a8e739 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1259 microversion: 2.1 time: 0.241234 Sep 04 01:09:25.509707 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-5e0601dc-545e-47e1-b534-e9249d1de4c8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:25.520425 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 359/736] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:25 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1259 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:09:25.520600 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5e0601dc-545e-47e1-b534-e9249d1de4c8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:25.520600 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5e0601dc-545e-47e1-b534-e9249d1de4c8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:25.520600 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5e0601dc-545e-47e1-b534-e9249d1de4c8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:25.520600 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5e0601dc-545e-47e1-b534-e9249d1de4c8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:25.520904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5e0601dc-545e-47e1-b534-e9249d1de4c8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:25.520904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5e0601dc-545e-47e1-b534-e9249d1de4c8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:25.608837 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.objects.instance [None req-5e0601dc-545e-47e1-b534-e9249d1de4c8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lazy-loading 'services' on Instance uuid 784b2b9b-9890-4f9e-be75-2548b520e138 {{(pid=121036) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:09:25.801987 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-5e0601dc-545e-47e1-b534-e9249d1de4c8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "DELETE /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138/os-volume_attachments/c8868d57-3f95-40ba-8bb5-0b47ba862842" status: 202 len: 0 microversion: 2.1 time: 0.297548 Sep 04 01:09:25.802602 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 378/737] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Sep 4 01:09:25 2022] DELETE /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138/os-volume_attachments/c8868d57-3f95-40ba-8bb5-0b47ba862842 => generated 0 bytes in 299 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 04 01:09:25.811511 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-57f10fe4-6f07-4cb5-8b6b-3a21926f523b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:25.813843 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-57f10fe4-6f07-4cb5-8b6b-3a21926f523b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:25.820120 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-57f10fe4-6f07-4cb5-8b6b-3a21926f523b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:25.820595 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-57f10fe4-6f07-4cb5-8b6b-3a21926f523b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:25.821031 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-57f10fe4-6f07-4cb5-8b6b-3a21926f523b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:25.821808 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-57f10fe4-6f07-4cb5-8b6b-3a21926f523b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:25.822258 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-57f10fe4-6f07-4cb5-8b6b-3a21926f523b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:25.822683 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-57f10fe4-6f07-4cb5-8b6b-3a21926f523b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:25.852602 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-57f10fe4-6f07-4cb5-8b6b-3a21926f523b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045075 Sep 04 01:09:25.853186 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 360/738] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 4 01:09:25 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:09:26.225895 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-26e3dce3-771a-4664-b476-b98a435f999b tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:26.226483 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-26e3dce3-771a-4664-b476-b98a435f999b tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: a3994b76-60e7-4d9a-a655-2fb09eb68a5d] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:26.232832 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-26e3dce3-771a-4664-b476-b98a435f999b tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:26.233354 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-26e3dce3-771a-4664-b476-b98a435f999b tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:26.240762 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-26e3dce3-771a-4664-b476-b98a435f999b tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:26.240762 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-26e3dce3-771a-4664-b476-b98a435f999b tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:26.240762 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-26e3dce3-771a-4664-b476-b98a435f999b tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:26.240762 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-26e3dce3-771a-4664-b476-b98a435f999b tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:26.260606 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-26e3dce3-771a-4664-b476-b98a435f999b tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: a3994b76-60e7-4d9a-a655-2fb09eb68a5d] Going to try to terminate instance {{(pid=121036) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 04 01:09:26.429576 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-83ad177c-3720-42a4-8291-ab4a86323f1b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:26.433666 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-83ad177c-3720-42a4-8291-ab4a86323f1b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:26.442117 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-83ad177c-3720-42a4-8291-ab4a86323f1b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:26.442772 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-83ad177c-3720-42a4-8291-ab4a86323f1b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:26.443244 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-83ad177c-3720-42a4-8291-ab4a86323f1b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:26.444987 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-83ad177c-3720-42a4-8291-ab4a86323f1b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:26.445515 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-83ad177c-3720-42a4-8291-ab4a86323f1b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:26.445951 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-83ad177c-3720-42a4-8291-ab4a86323f1b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:26.519320 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-26e3dce3-771a-4664-b476-b98a435f999b tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "DELETE /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d" status: 204 len: 0 microversion: 2.1 time: 0.297819 Sep 04 01:09:26.520207 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 379/739] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 4 01:09:26 2022] DELETE /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d => generated 0 bytes in 300 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 04 01:09:26.529692 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-17db74a2-23f3-4b8d-9e01-b354d4ad6a0f tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:26.530522 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-17db74a2-23f3-4b8d-9e01-b354d4ad6a0f tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: a3994b76-60e7-4d9a-a655-2fb09eb68a5d] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:26.537274 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-17db74a2-23f3-4b8d-9e01-b354d4ad6a0f tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:26.537916 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-17db74a2-23f3-4b8d-9e01-b354d4ad6a0f tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:26.538359 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-17db74a2-23f3-4b8d-9e01-b354d4ad6a0f tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:26.539061 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-17db74a2-23f3-4b8d-9e01-b354d4ad6a0f tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:26.539513 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-17db74a2-23f3-4b8d-9e01-b354d4ad6a0f tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:26.539941 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-17db74a2-23f3-4b8d-9e01-b354d4ad6a0f tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:26.616026 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-83ad177c-3720-42a4-8291-ab4a86323f1b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:26.624658 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-83ad177c-3720-42a4-8291-ab4a86323f1b tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1735 microversion: 2.1 time: 0.200556 Sep 04 01:09:26.625262 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 361/740] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:26 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1735 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:09:26.630029 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-c99d52d0-f42e-4cbe-b9af-b6438991243e tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:26.631466 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-c99d52d0-f42e-4cbe-b9af-b6438991243e tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:26.638139 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c99d52d0-f42e-4cbe-b9af-b6438991243e tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:26.638623 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c99d52d0-f42e-4cbe-b9af-b6438991243e tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:26.639052 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c99d52d0-f42e-4cbe-b9af-b6438991243e tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:26.639903 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c99d52d0-f42e-4cbe-b9af-b6438991243e tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:26.640360 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c99d52d0-f42e-4cbe-b9af-b6438991243e tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:26.640781 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c99d52d0-f42e-4cbe-b9af-b6438991243e tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:26.673987 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-17db74a2-23f3-4b8d-9e01-b354d4ad6a0f tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:26.682552 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-17db74a2-23f3-4b8d-9e01-b354d4ad6a0f tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d" status: 200 len: 1514 microversion: 2.1 time: 0.158007 Sep 04 01:09:26.683371 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 380/741] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:26 2022] GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d => generated 1514 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:26.770244 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-c99d52d0-f42e-4cbe-b9af-b6438991243e tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:26.777960 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-c99d52d0-f42e-4cbe-b9af-b6438991243e tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1259 microversion: 2.1 time: 0.151545 Sep 04 01:09:26.778527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 362/742] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:26 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1259 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:26.866041 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-4d50a1f9-ae2f-403c-9268-f7a6dec4b160 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:26.868389 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-4d50a1f9-ae2f-403c-9268-f7a6dec4b160 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:26.874398 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4d50a1f9-ae2f-403c-9268-f7a6dec4b160 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:26.874878 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4d50a1f9-ae2f-403c-9268-f7a6dec4b160 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:26.875316 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4d50a1f9-ae2f-403c-9268-f7a6dec4b160 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:26.876001 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4d50a1f9-ae2f-403c-9268-f7a6dec4b160 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:26.876454 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4d50a1f9-ae2f-403c-9268-f7a6dec4b160 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:26.876882 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4d50a1f9-ae2f-403c-9268-f7a6dec4b160 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:26.909878 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-4d50a1f9-ae2f-403c-9268-f7a6dec4b160 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048775 Sep 04 01:09:26.910440 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 381/743] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 4 01:09:26 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:09:27.649321 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-24e2b6f7-04d0-4892-8744-1b542efbcf48 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:27.649930 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-24e2b6f7-04d0-4892-8744-1b542efbcf48 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:27.656370 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-24e2b6f7-04d0-4892-8744-1b542efbcf48 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:27.656849 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-24e2b6f7-04d0-4892-8744-1b542efbcf48 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:27.657323 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-24e2b6f7-04d0-4892-8744-1b542efbcf48 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:27.658012 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-24e2b6f7-04d0-4892-8744-1b542efbcf48 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:27.658467 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-24e2b6f7-04d0-4892-8744-1b542efbcf48 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:27.658889 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-24e2b6f7-04d0-4892-8744-1b542efbcf48 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:27.706400 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-4a2d92f3-048a-4828-bfc6-177a5fb845bd tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:27.707002 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-4a2d92f3-048a-4828-bfc6-177a5fb845bd tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: a3994b76-60e7-4d9a-a655-2fb09eb68a5d] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:27.714301 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4a2d92f3-048a-4828-bfc6-177a5fb845bd tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:27.714720 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4a2d92f3-048a-4828-bfc6-177a5fb845bd tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:27.715159 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4a2d92f3-048a-4828-bfc6-177a5fb845bd tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:27.715877 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4a2d92f3-048a-4828-bfc6-177a5fb845bd tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:27.716326 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4a2d92f3-048a-4828-bfc6-177a5fb845bd tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:27.716744 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4a2d92f3-048a-4828-bfc6-177a5fb845bd tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:27.790643 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-24e2b6f7-04d0-4892-8744-1b542efbcf48 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:27.799187 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-24e2b6f7-04d0-4892-8744-1b542efbcf48 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1735 microversion: 2.1 time: 0.154650 Sep 04 01:09:27.799767 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 363/744] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:27 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1735 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:27.804485 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-33f5905d-a5d0-4743-9f78-9fc0f7565be1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:27.805115 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-33f5905d-a5d0-4743-9f78-9fc0f7565be1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:27.810911 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-33f5905d-a5d0-4743-9f78-9fc0f7565be1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:27.811392 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-33f5905d-a5d0-4743-9f78-9fc0f7565be1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:27.811941 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-33f5905d-a5d0-4743-9f78-9fc0f7565be1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:27.812638 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-33f5905d-a5d0-4743-9f78-9fc0f7565be1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:27.813357 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-33f5905d-a5d0-4743-9f78-9fc0f7565be1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:27.813855 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-33f5905d-a5d0-4743-9f78-9fc0f7565be1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:27.847960 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-4a2d92f3-048a-4828-bfc6-177a5fb845bd tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:27.855833 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-4a2d92f3-048a-4828-bfc6-177a5fb845bd tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d" status: 200 len: 1514 microversion: 2.1 time: 0.153299 Sep 04 01:09:27.856410 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 382/745] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:27 2022] GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d => generated 1514 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:09:27.923059 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b5733efb-21b9-4f24-8cda-d1faf694f08b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:27.925427 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-b5733efb-21b9-4f24-8cda-d1faf694f08b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:27.931917 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b5733efb-21b9-4f24-8cda-d1faf694f08b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:27.932402 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b5733efb-21b9-4f24-8cda-d1faf694f08b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:27.932837 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b5733efb-21b9-4f24-8cda-d1faf694f08b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:27.933583 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b5733efb-21b9-4f24-8cda-d1faf694f08b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:27.934037 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b5733efb-21b9-4f24-8cda-d1faf694f08b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:27.934461 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b5733efb-21b9-4f24-8cda-d1faf694f08b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:27.946704 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-33f5905d-a5d0-4743-9f78-9fc0f7565be1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:27.954696 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-33f5905d-a5d0-4743-9f78-9fc0f7565be1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1259 microversion: 2.1 time: 0.153941 Sep 04 01:09:27.955264 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 364/746] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:27 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1259 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:27.966470 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-b5733efb-21b9-4f24-8cda-d1faf694f08b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047275 Sep 04 01:09:27.967101 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 383/747] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 4 01:09:27 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:09:28.822611 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-c0c0aa96-7d35-40c7-95fa-eeee3a3628b6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:28.823298 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-c0c0aa96-7d35-40c7-95fa-eeee3a3628b6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:28.829692 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c0c0aa96-7d35-40c7-95fa-eeee3a3628b6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:28.830176 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c0c0aa96-7d35-40c7-95fa-eeee3a3628b6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:28.830613 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c0c0aa96-7d35-40c7-95fa-eeee3a3628b6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:28.831305 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c0c0aa96-7d35-40c7-95fa-eeee3a3628b6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:28.831786 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c0c0aa96-7d35-40c7-95fa-eeee3a3628b6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:28.832215 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c0c0aa96-7d35-40c7-95fa-eeee3a3628b6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:28.879863 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-64a1f66e-8d5d-44a2-8d0d-0fc81b274e70 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:28.880602 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-64a1f66e-8d5d-44a2-8d0d-0fc81b274e70 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: a3994b76-60e7-4d9a-a655-2fb09eb68a5d] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:28.887899 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-64a1f66e-8d5d-44a2-8d0d-0fc81b274e70 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:28.888381 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-64a1f66e-8d5d-44a2-8d0d-0fc81b274e70 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:28.888816 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-64a1f66e-8d5d-44a2-8d0d-0fc81b274e70 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:28.889569 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-64a1f66e-8d5d-44a2-8d0d-0fc81b274e70 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:28.890143 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-64a1f66e-8d5d-44a2-8d0d-0fc81b274e70 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:28.890592 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-64a1f66e-8d5d-44a2-8d0d-0fc81b274e70 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:28.966646 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-c0c0aa96-7d35-40c7-95fa-eeee3a3628b6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:28.976039 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-c0c0aa96-7d35-40c7-95fa-eeee3a3628b6 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1735 microversion: 2.1 time: 0.157876 Sep 04 01:09:28.976839 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 365/748] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:28 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1735 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:28.981761 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-e3377278-1b1d-4fc7-83f7-11da3a00d2a0 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:28.982539 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-e3377278-1b1d-4fc7-83f7-11da3a00d2a0 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:28.989387 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e3377278-1b1d-4fc7-83f7-11da3a00d2a0 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:28.990576 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e3377278-1b1d-4fc7-83f7-11da3a00d2a0 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:28.991168 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e3377278-1b1d-4fc7-83f7-11da3a00d2a0 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:28.992128 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e3377278-1b1d-4fc7-83f7-11da3a00d2a0 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:28.992742 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e3377278-1b1d-4fc7-83f7-11da3a00d2a0 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:28.993204 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e3377278-1b1d-4fc7-83f7-11da3a00d2a0 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:29.024377 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-64a1f66e-8d5d-44a2-8d0d-0fc81b274e70 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:29.032295 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-64a1f66e-8d5d-44a2-8d0d-0fc81b274e70 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d" status: 200 len: 1514 microversion: 2.1 time: 0.156446 Sep 04 01:09:29.032863 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 384/749] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:28 2022] GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d => generated 1514 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:09:29.037919 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-f884c685-5944-4cc6-b387-6460537da837 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:29.040141 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-f884c685-5944-4cc6-b387-6460537da837 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:29.046002 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f884c685-5944-4cc6-b387-6460537da837 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:29.046485 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f884c685-5944-4cc6-b387-6460537da837 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:29.046919 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f884c685-5944-4cc6-b387-6460537da837 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:29.047614 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f884c685-5944-4cc6-b387-6460537da837 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:29.048067 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f884c685-5944-4cc6-b387-6460537da837 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:29.048493 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f884c685-5944-4cc6-b387-6460537da837 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:29.079228 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-f884c685-5944-4cc6-b387-6460537da837 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045310 Sep 04 01:09:29.079962 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 385/750] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 4 01:09:29 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:09:29.128051 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-e3377278-1b1d-4fc7-83f7-11da3a00d2a0 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:29.135908 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-e3377278-1b1d-4fc7-83f7-11da3a00d2a0 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1259 microversion: 2.1 time: 0.158037 Sep 04 01:09:29.136474 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 366/751] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:28 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1259 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:30.005390 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-5ccbff3f-d5e0-4ae1-9a28-0d3a9abe06a8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:30.006697 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-5ccbff3f-d5e0-4ae1-9a28-0d3a9abe06a8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:30.014053 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5ccbff3f-d5e0-4ae1-9a28-0d3a9abe06a8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:30.014622 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5ccbff3f-d5e0-4ae1-9a28-0d3a9abe06a8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:30.015109 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5ccbff3f-d5e0-4ae1-9a28-0d3a9abe06a8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:30.015926 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5ccbff3f-d5e0-4ae1-9a28-0d3a9abe06a8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:30.016423 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5ccbff3f-d5e0-4ae1-9a28-0d3a9abe06a8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:30.016890 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5ccbff3f-d5e0-4ae1-9a28-0d3a9abe06a8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:30.021313 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=3,nova_cell1:UPDATE=1 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:09:30.057416 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-4aa9e4f4-1bf3-4d11-aa2d-6b873dd737a7 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:30.058131 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-4aa9e4f4-1bf3-4d11-aa2d-6b873dd737a7 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: a3994b76-60e7-4d9a-a655-2fb09eb68a5d] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:30.068059 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4aa9e4f4-1bf3-4d11-aa2d-6b873dd737a7 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:30.069652 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4aa9e4f4-1bf3-4d11-aa2d-6b873dd737a7 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:30.070285 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4aa9e4f4-1bf3-4d11-aa2d-6b873dd737a7 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:30.071135 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4aa9e4f4-1bf3-4d11-aa2d-6b873dd737a7 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:30.071623 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4aa9e4f4-1bf3-4d11-aa2d-6b873dd737a7 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:30.072247 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4aa9e4f4-1bf3-4d11-aa2d-6b873dd737a7 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:30.171070 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-5ccbff3f-d5e0-4ae1-9a28-0d3a9abe06a8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:30.179926 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-5ccbff3f-d5e0-4ae1-9a28-0d3a9abe06a8 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1735 microversion: 2.1 time: 0.181603 Sep 04 01:09:30.180493 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 386/752] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:29 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1735 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:30.185477 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-e3946a45-b99d-4a97-96e2-3e865b47b711 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:30.187820 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-e3946a45-b99d-4a97-96e2-3e865b47b711 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:30.195409 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e3946a45-b99d-4a97-96e2-3e865b47b711 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:30.195891 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e3946a45-b99d-4a97-96e2-3e865b47b711 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:30.196536 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e3946a45-b99d-4a97-96e2-3e865b47b711 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:30.197535 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e3946a45-b99d-4a97-96e2-3e865b47b711 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:30.198138 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e3946a45-b99d-4a97-96e2-3e865b47b711 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:30.198711 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e3946a45-b99d-4a97-96e2-3e865b47b711 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:30.230104 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-e3946a45-b99d-4a97-96e2-3e865b47b711 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048511 Sep 04 01:09:30.230778 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 387/753] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 4 01:09:30 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:09:30.235344 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-550d0ecc-2149-4d58-9784-1fa41aa1bcb6 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:30.236353 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-550d0ecc-2149-4d58-9784-1fa41aa1bcb6 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:30.243450 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-550d0ecc-2149-4d58-9784-1fa41aa1bcb6 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:30.243933 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-550d0ecc-2149-4d58-9784-1fa41aa1bcb6 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:30.244366 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-550d0ecc-2149-4d58-9784-1fa41aa1bcb6 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:30.245050 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-550d0ecc-2149-4d58-9784-1fa41aa1bcb6 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:30.245535 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-550d0ecc-2149-4d58-9784-1fa41aa1bcb6 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:30.245995 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-550d0ecc-2149-4d58-9784-1fa41aa1bcb6 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:30.333285 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-4aa9e4f4-1bf3-4d11-aa2d-6b873dd737a7 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:30.343288 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-4aa9e4f4-1bf3-4d11-aa2d-6b873dd737a7 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d" status: 200 len: 1514 microversion: 2.1 time: 0.290884 Sep 04 01:09:30.344445 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 367/754] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:30 2022] GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d => generated 1514 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:30.393753 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-550d0ecc-2149-4d58-9784-1fa41aa1bcb6 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:30.402133 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-550d0ecc-2149-4d58-9784-1fa41aa1bcb6 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1259 microversion: 2.1 time: 0.170098 Sep 04 01:09:30.402725 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 388/755] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:30 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1259 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:30.734554 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-fb89f8d9-7800-40a3-8815-a7610b502ae0 req-747740b4-e2cb-4635-80d7-c73e41434995 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a3994b76-60e7-4d9a-a655-2fb09eb68a5d", "name": "network-vif-deleted", "tag": "88ae446d-0ce0-4187-b028-5915cabba54c"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:09:30.745030 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-fb89f8d9-7800-40a3-8815-a7610b502ae0 req-747740b4-e2cb-4635-80d7-c73e41434995 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:30.745573 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-fb89f8d9-7800-40a3-8815-a7610b502ae0 req-747740b4-e2cb-4635-80d7-c73e41434995 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:30.750727 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-fb89f8d9-7800-40a3-8815-a7610b502ae0 req-747740b4-e2cb-4635-80d7-c73e41434995 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:30.750727 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=100 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:09:30.757407 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-fb89f8d9-7800-40a3-8815-a7610b502ae0 req-747740b4-e2cb-4635-80d7-c73e41434995 service nova] Creating event network-vif-deleted:88ae446d-0ce0-4187-b028-5915cabba54c for instance a3994b76-60e7-4d9a-a655-2fb09eb68a5d on hv-cinder-maste Sep 04 01:09:30.762512 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-fb89f8d9-7800-40a3-8815-a7610b502ae0 req-747740b4-e2cb-4635-80d7-c73e41434995 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031586 Sep 04 01:09:30.763149 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 368/756] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:09:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:09:31.205604 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-f7dffd72-45b8-4782-9e3a-97a04a871882 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:31.206194 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-f7dffd72-45b8-4782-9e3a-97a04a871882 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:31.213360 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f7dffd72-45b8-4782-9e3a-97a04a871882 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:31.213840 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f7dffd72-45b8-4782-9e3a-97a04a871882 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:31.214272 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f7dffd72-45b8-4782-9e3a-97a04a871882 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:31.214950 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f7dffd72-45b8-4782-9e3a-97a04a871882 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:31.215422 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f7dffd72-45b8-4782-9e3a-97a04a871882 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:31.215844 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f7dffd72-45b8-4782-9e3a-97a04a871882 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:31.244738 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-53115280-a3ab-44c4-a770-b0b8deaa3ac5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:31.247039 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-53115280-a3ab-44c4-a770-b0b8deaa3ac5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:31.253340 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-53115280-a3ab-44c4-a770-b0b8deaa3ac5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:31.253994 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-53115280-a3ab-44c4-a770-b0b8deaa3ac5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:31.254435 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-53115280-a3ab-44c4-a770-b0b8deaa3ac5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:31.255131 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-53115280-a3ab-44c4-a770-b0b8deaa3ac5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:31.255578 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-53115280-a3ab-44c4-a770-b0b8deaa3ac5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:31.256506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-53115280-a3ab-44c4-a770-b0b8deaa3ac5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:31.288730 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-53115280-a3ab-44c4-a770-b0b8deaa3ac5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.049781 Sep 04 01:09:31.289541 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 369/757] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 4 01:09:31 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138/os-volume_attachments => generated 25 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Sep 04 01:09:31.350123 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-f7dffd72-45b8-4782-9e3a-97a04a871882 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:31.356367 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-8a58d63a-f3a4-4be9-bf23-9723125cd409 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:31.356882 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-8a58d63a-f3a4-4be9-bf23-9723125cd409 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:31.359729 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-f7dffd72-45b8-4782-9e3a-97a04a871882 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1715 microversion: 2.1 time: 0.158123 Sep 04 01:09:31.360501 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 389/758] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:31 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1715 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:31.365829 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8a58d63a-f3a4-4be9-bf23-9723125cd409 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:31.374154 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8a58d63a-f3a4-4be9-bf23-9723125cd409 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:31.374154 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8a58d63a-f3a4-4be9-bf23-9723125cd409 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:31.374491 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-4330865b-0549-4026-b6f5-87b93f6ebea1 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:31.374684 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8a58d63a-f3a4-4be9-bf23-9723125cd409 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:31.374779 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-4330865b-0549-4026-b6f5-87b93f6ebea1 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: a3994b76-60e7-4d9a-a655-2fb09eb68a5d] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:31.374951 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8a58d63a-f3a4-4be9-bf23-9723125cd409 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:31.374951 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8a58d63a-f3a4-4be9-bf23-9723125cd409 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:31.375148 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4330865b-0549-4026-b6f5-87b93f6ebea1 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:31.375148 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4330865b-0549-4026-b6f5-87b93f6ebea1 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:31.375460 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4330865b-0549-4026-b6f5-87b93f6ebea1 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:31.376145 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4330865b-0549-4026-b6f5-87b93f6ebea1 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:31.376588 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4330865b-0549-4026-b6f5-87b93f6ebea1 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:31.377007 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4330865b-0549-4026-b6f5-87b93f6ebea1 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:31.408571 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-8a58d63a-f3a4-4be9-bf23-9723125cd409 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Going to try to terminate instance {{(pid=121037) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 04 01:09:31.414677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4330865b-0549-4026-b6f5-87b93f6ebea1 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:31.415161 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4330865b-0549-4026-b6f5-87b93f6ebea1 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:31.415593 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4330865b-0549-4026-b6f5-87b93f6ebea1 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:31.416041 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.objects.instance [None req-4330865b-0549-4026-b6f5-87b93f6ebea1 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lazy-loading 'fault' on Instance uuid a3994b76-60e7-4d9a-a655-2fb09eb68a5d {{(pid=121036) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:09:31.472151 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-4330865b-0549-4026-b6f5-87b93f6ebea1 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:31.481030 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-4330865b-0549-4026-b6f5-87b93f6ebea1 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d" status: 200 len: 1309 microversion: 2.1 time: 0.117835 Sep 04 01:09:31.481661 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 390/759] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:31 2022] GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d => generated 1309 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:31.487348 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-ebfe42cd-b83c-4847-93bd-1f047eafad89 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:31.488310 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-ebfe42cd-b83c-4847-93bd-1f047eafad89 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:31.494919 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ebfe42cd-b83c-4847-93bd-1f047eafad89 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:31.495587 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ebfe42cd-b83c-4847-93bd-1f047eafad89 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:31.496170 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ebfe42cd-b83c-4847-93bd-1f047eafad89 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:31.497265 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ebfe42cd-b83c-4847-93bd-1f047eafad89 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:31.497960 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ebfe42cd-b83c-4847-93bd-1f047eafad89 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:31.498382 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ebfe42cd-b83c-4847-93bd-1f047eafad89 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:31.632845 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-8a58d63a-f3a4-4be9-bf23-9723125cd409 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "DELETE /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 204 len: 0 microversion: 2.1 time: 0.279858 Sep 04 01:09:31.633119 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ebfe42cd-b83c-4847-93bd-1f047eafad89 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:31.633465 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 370/760] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 4 01:09:31 2022] DELETE /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 04 01:09:31.638645 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-6ac547ba-0fc5-4b00-abe2-9a3843c541ff tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:31.639219 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-6ac547ba-0fc5-4b00-abe2-9a3843c541ff tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:31.641508 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-ebfe42cd-b83c-4847-93bd-1f047eafad89 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1259 microversion: 2.1 time: 0.158587 Sep 04 01:09:31.642109 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 391/761] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:31 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1259 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:31.645296 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6ac547ba-0fc5-4b00-abe2-9a3843c541ff tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:31.645780 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6ac547ba-0fc5-4b00-abe2-9a3843c541ff tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:31.646213 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6ac547ba-0fc5-4b00-abe2-9a3843c541ff tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:31.647162 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-a62ddae7-9179-479d-8474-e5d0276faef7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:31.647342 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6ac547ba-0fc5-4b00-abe2-9a3843c541ff tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:31.647644 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6ac547ba-0fc5-4b00-abe2-9a3843c541ff tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:31.647768 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-a62ddae7-9179-479d-8474-e5d0276faef7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:31.648091 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6ac547ba-0fc5-4b00-abe2-9a3843c541ff tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:31.653765 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a62ddae7-9179-479d-8474-e5d0276faef7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:31.654777 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a62ddae7-9179-479d-8474-e5d0276faef7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:31.655138 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a62ddae7-9179-479d-8474-e5d0276faef7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:31.655851 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a62ddae7-9179-479d-8474-e5d0276faef7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:31.656623 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a62ddae7-9179-479d-8474-e5d0276faef7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:31.657057 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a62ddae7-9179-479d-8474-e5d0276faef7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:31.778339 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.wsgi [None req-6ac547ba-0fc5-4b00-abe2-9a3843c541ff tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] HTTP exception thrown: Cannot 'detach_volume' instance 2570ee6c-f335-41bc-8e67-f89a62b28134 while it is in vm_state rescued Sep 04 01:09:31.780205 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-6ac547ba-0fc5-4b00-abe2-9a3843c541ff tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Request method failure captured: Sep 04 01:09:31.780205 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: request: DELETE /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134/os-volume_attachments/10a678c8-9649-469a-b1ef-7e0530ce26b8 HTTP/1.1 Sep 04 01:09:31.780205 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept: application/json Sep 04 01:09:31.780205 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept-Encoding: identity Sep 04 01:09:31.780205 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Connection: close Sep 04 01:09:31.780205 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Length: 0 Sep 04 01:09:31.780205 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Type: application/json Sep 04 01:09:31.780205 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Host: 10.222.0.42 Sep 04 01:09:31.780205 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Openstack-System-Scope: None Sep 04 01:09:31.780205 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: User-Agent: python-urllib3/1.26.12 Sep 04 01:09:31.780205 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Auth-Token: gAAAAABjE_nMCq4zD8BmB6mF8h26w0y_IFei-x7tMbI5vcKfWFaeAi75nOGGI72_ZHVfHvHOCK5qrlGll_NneJYvZkWgGCkELi_QIwkIIEFfz91_ouiYPU3bkvjmjKoIsPt_9g8qfiCyLaERnndIFBkeIUe0g0NvmODNU753JT7NKZpvzaTLcsI Sep 04 01:09:31.780205 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Id: None Sep 04 01:09:31.780205 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Name: None Sep 04 01:09:31.780205 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Identity-Status: Confirmed Sep 04 01:09:31.780205 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Is-Admin-Project: True Sep 04 01:09:31.780205 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Id: default Sep 04 01:09:31.780205 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Name: Default Sep 04 01:09:31.780205 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Id: db72b63281384bd883b04d05bcc35677 Sep 04 01:09:31.780205 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1367456352 Sep 04 01:09:31.780205 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Role: reader,member Sep 04 01:09:31.780205 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Roles: reader,member Sep 04 01:09:31.781712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/db72b63281384bd883b04d05bcc35677"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_db72b63281384bd883b04d05bcc35677", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/db72b63281384bd883b04d05bcc35677"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/db72b63281384bd883b04d05bcc35677"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:09:31.781712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1367456352 Sep 04 01:09:31.781712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Id: db72b63281384bd883b04d05bcc35677 Sep 04 01:09:31.781712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1367456352 Sep 04 01:09:31.781712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User: tempest-ServerRescueNegativeTestJSON-1367456352-project Sep 04 01:09:31.781712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Id: default Sep 04 01:09:31.781712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Name: Default Sep 04 01:09:31.781712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Id: bd13df3a2d0e4182bbd86dab7035c371 Sep 04 01:09:31.781712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1367456352-project Sep 04 01:09:31.781712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: method: > Sep 04 01:09:31.781712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: exception: Cannot 'detach_volume' instance 2570ee6c-f335-41bc-8e67-f89a62b28134 while it is in vm_state rescued Sep 04 01:09:31.782491 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: action_args: {'server_id': '2570ee6c-f335-41bc-8e67-f89a62b28134', 'id': '10a678c8-9649-469a-b1ef-7e0530ce26b8'} Sep 04 01:09:31.782491 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:09:31.782491 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:09:31.782491 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 04 01:09:31.782491 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 04 01:09:31.782491 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 04 01:09:31.782491 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 04 01:09:31.782491 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Sep 04 01:09:31.782491 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 04 01:09:31.782491 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 2570ee6c-f335-41bc-8e67-f89a62b28134 in vm_state rescued. Cannot detach_volume while the instance is in this state. Sep 04 01:09:31.782491 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:09:31.782491 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:09:31.782491 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:09:31.782491 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:09:31.782491 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:09:31.782491 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:09:31.782491 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:09:31.782491 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:09:31.782491 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:09:31.782491 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:09:31.783912 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Sep 04 01:09:31.783912 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Sep 04 01:09:31.783912 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 04 01:09:31.783912 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 04 01:09:31.783912 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 2570ee6c-f335-41bc-8e67-f89a62b28134 while it is in vm_state rescued Sep 04 01:09:31.783912 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:09:31.783912 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:09:31.783912 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:09:31.783912 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:09:31.783912 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:09:31.783912 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:09:31.783912 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:09:31.783912 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:09:31.783912 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 2570ee6c-f335-41bc-8e67-f89a62b28134 while it is in vm_state rescued Sep 04 01:09:31.783912 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:09:31.783912 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-6ac547ba-0fc5-4b00-abe2-9a3843c541ff tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Returning 409 to user: Cannot 'detach_volume' instance 2570ee6c-f335-41bc-8e67-f89a62b28134 while it is in vm_state rescued {{(pid=121037) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:09:31.784955 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-6ac547ba-0fc5-4b00-abe2-9a3843c541ff tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "DELETE /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134/os-volume_attachments/10a678c8-9649-469a-b1ef-7e0530ce26b8" status: 409 len: 152 microversion: 2.1 time: 0.147294 Sep 04 01:09:31.784955 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 371/762] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Sep 4 01:09:31 2022] DELETE /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134/os-volume_attachments/10a678c8-9649-469a-b1ef-7e0530ce26b8 => generated 152 bytes in 149 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Sep 04 01:09:31.791075 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-71ab0e5c-6e63-419b-aba5-5146e51ad5c3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:09:31.791600 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-71ab0e5c-6e63-419b-aba5-5146e51ad5c3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:31.791744 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-a62ddae7-9179-479d-8474-e5d0276faef7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:31.798233 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-71ab0e5c-6e63-419b-aba5-5146e51ad5c3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:31.798720 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-71ab0e5c-6e63-419b-aba5-5146e51ad5c3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:31.799160 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-71ab0e5c-6e63-419b-aba5-5146e51ad5c3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:31.799580 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-a62ddae7-9179-479d-8474-e5d0276faef7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1684 microversion: 2.1 time: 0.156365 Sep 04 01:09:31.799852 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-71ab0e5c-6e63-419b-aba5-5146e51ad5c3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:31.800153 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 392/763] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:31 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1684 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:31.800359 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-71ab0e5c-6e63-419b-aba5-5146e51ad5c3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:31.800735 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-71ab0e5c-6e63-419b-aba5-5146e51ad5c3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:31.978198 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-71ab0e5c-6e63-419b-aba5-5146e51ad5c3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "POST /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134/action" status: 202 len: 0 microversion: 2.1 time: 0.190738 Sep 04 01:09:31.978780 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 372/764] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Sep 4 01:09:31 2022] POST /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134/action => generated 0 bytes in 192 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 04 01:09:31.987073 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-974e1f13-0e29-4659-b0c1-9bf4b7e8d111 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:31.987683 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-974e1f13-0e29-4659-b0c1-9bf4b7e8d111 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:31.994415 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-974e1f13-0e29-4659-b0c1-9bf4b7e8d111 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:31.994894 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-974e1f13-0e29-4659-b0c1-9bf4b7e8d111 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:31.995327 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-974e1f13-0e29-4659-b0c1-9bf4b7e8d111 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:31.996010 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-974e1f13-0e29-4659-b0c1-9bf4b7e8d111 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:31.996463 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-974e1f13-0e29-4659-b0c1-9bf4b7e8d111 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:31.996887 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-974e1f13-0e29-4659-b0c1-9bf4b7e8d111 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:32.361793 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-974e1f13-0e29-4659-b0c1-9bf4b7e8d111 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:32.370574 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-974e1f13-0e29-4659-b0c1-9bf4b7e8d111 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1723 microversion: 2.1 time: 0.387617 Sep 04 01:09:32.371145 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 393/765] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:31 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1723 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:32.506048 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-28c98b0e-b0fb-4bb4-955d-e33d7bd8bbcb tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:32.506838 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-28c98b0e-b0fb-4bb4-955d-e33d7bd8bbcb tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: a3994b76-60e7-4d9a-a655-2fb09eb68a5d] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:32.513097 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-28c98b0e-b0fb-4bb4-955d-e33d7bd8bbcb tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:32.513586 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-28c98b0e-b0fb-4bb4-955d-e33d7bd8bbcb tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:32.514030 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-28c98b0e-b0fb-4bb4-955d-e33d7bd8bbcb tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:32.514795 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-28c98b0e-b0fb-4bb4-955d-e33d7bd8bbcb tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:32.515253 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-28c98b0e-b0fb-4bb4-955d-e33d7bd8bbcb tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:32.520617 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-28c98b0e-b0fb-4bb4-955d-e33d7bd8bbcb tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:32.547848 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-28c98b0e-b0fb-4bb4-955d-e33d7bd8bbcb tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:32.548323 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-28c98b0e-b0fb-4bb4-955d-e33d7bd8bbcb tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:32.548755 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-28c98b0e-b0fb-4bb4-955d-e33d7bd8bbcb tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:32.549227 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.objects.instance [None req-28c98b0e-b0fb-4bb4-955d-e33d7bd8bbcb tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lazy-loading 'fault' on Instance uuid a3994b76-60e7-4d9a-a655-2fb09eb68a5d {{(pid=121037) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:09:32.589809 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-28c98b0e-b0fb-4bb4-955d-e33d7bd8bbcb tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '77de7bb68f3c45d680c8a78dc40df07b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '6071c3a795ba4aa098e3af8cde306d58', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:32.597328 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-28c98b0e-b0fb-4bb4-955d-e33d7bd8bbcb tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d" status: 200 len: 1309 microversion: 2.1 time: 0.095675 Sep 04 01:09:32.597885 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 373/766] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:32 2022] GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d => generated 1309 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:32.664419 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-3f518cef-1705-4da0-832d-a67e2c22b7c1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:32.665322 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-3f518cef-1705-4da0-832d-a67e2c22b7c1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:32.671266 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3f518cef-1705-4da0-832d-a67e2c22b7c1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:32.671745 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3f518cef-1705-4da0-832d-a67e2c22b7c1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:32.672175 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3f518cef-1705-4da0-832d-a67e2c22b7c1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:32.672858 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3f518cef-1705-4da0-832d-a67e2c22b7c1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:32.673326 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3f518cef-1705-4da0-832d-a67e2c22b7c1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:32.673750 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3f518cef-1705-4da0-832d-a67e2c22b7c1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:32.801032 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-3f518cef-1705-4da0-832d-a67e2c22b7c1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:32.808776 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-3f518cef-1705-4da0-832d-a67e2c22b7c1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1259 microversion: 2.1 time: 0.148885 Sep 04 01:09:32.809375 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 394/767] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:32 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1259 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:32.824308 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-24c02845-2321-4d63-9d6a-3ff72bafe6e8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:32.824907 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-24c02845-2321-4d63-9d6a-3ff72bafe6e8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:32.830823 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-24c02845-2321-4d63-9d6a-3ff72bafe6e8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:32.831296 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-24c02845-2321-4d63-9d6a-3ff72bafe6e8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:32.831727 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-24c02845-2321-4d63-9d6a-3ff72bafe6e8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:32.832403 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-24c02845-2321-4d63-9d6a-3ff72bafe6e8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:32.832844 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-24c02845-2321-4d63-9d6a-3ff72bafe6e8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:32.833297 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-24c02845-2321-4d63-9d6a-3ff72bafe6e8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:33.202412 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-24c02845-2321-4d63-9d6a-3ff72bafe6e8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:33.210405 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-24c02845-2321-4d63-9d6a-3ff72bafe6e8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1684 microversion: 2.1 time: 0.389897 Sep 04 01:09:33.210975 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 374/768] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:32 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1684 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:33.393832 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-8cadb316-2221-4855-90cf-7901d9eb49c3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:33.394486 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-8cadb316-2221-4855-90cf-7901d9eb49c3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:33.400848 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8cadb316-2221-4855-90cf-7901d9eb49c3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:33.404967 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8cadb316-2221-4855-90cf-7901d9eb49c3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:33.404967 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8cadb316-2221-4855-90cf-7901d9eb49c3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:33.404967 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8cadb316-2221-4855-90cf-7901d9eb49c3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:33.404967 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8cadb316-2221-4855-90cf-7901d9eb49c3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:33.405335 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8cadb316-2221-4855-90cf-7901d9eb49c3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:33.538750 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-8cadb316-2221-4855-90cf-7901d9eb49c3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:33.546950 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-8cadb316-2221-4855-90cf-7901d9eb49c3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1723 microversion: 2.1 time: 0.157031 Sep 04 01:09:33.547520 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 395/769] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:33 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1723 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:33.621626 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-e65d0be6-b28a-40f6-b559-2fd1b573a860 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:33.622247 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-e65d0be6-b28a-40f6-b559-2fd1b573a860 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] [instance: a3994b76-60e7-4d9a-a655-2fb09eb68a5d] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:33.628342 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e65d0be6-b28a-40f6-b559-2fd1b573a860 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:33.628954 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e65d0be6-b28a-40f6-b559-2fd1b573a860 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:33.629428 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e65d0be6-b28a-40f6-b559-2fd1b573a860 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:33.630117 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e65d0be6-b28a-40f6-b559-2fd1b573a860 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:33.630564 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e65d0be6-b28a-40f6-b559-2fd1b573a860 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:33.630986 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e65d0be6-b28a-40f6-b559-2fd1b573a860 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:33.644517 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.wsgi [None req-e65d0be6-b28a-40f6-b559-2fd1b573a860 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] HTTP exception thrown: Instance a3994b76-60e7-4d9a-a655-2fb09eb68a5d could not be found. Sep 04 01:09:33.646141 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-e65d0be6-b28a-40f6-b559-2fd1b573a860 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Request method failure captured: Sep 04 01:09:33.646141 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: request: GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d HTTP/1.1 Sep 04 01:09:33.646141 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept: application/json Sep 04 01:09:33.646141 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept-Encoding: identity Sep 04 01:09:33.646141 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Connection: close Sep 04 01:09:33.646141 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Length: 0 Sep 04 01:09:33.646141 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Type: application/json Sep 04 01:09:33.646141 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Host: 10.222.0.42 Sep 04 01:09:33.646141 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Openstack-System-Scope: None Sep 04 01:09:33.646141 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: User-Agent: python-urllib3/1.26.12 Sep 04 01:09:33.646141 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Auth-Token: gAAAAABjE_obhYA9_bHoxMJ9YiLa5KIN-wLSWbgj9mawygfSrhTusnb451gJNWZloutBmVTPKPhC2n3ys4WuBDIXKrrGT73hy4If6MlaTXkCBJYE0CckFsyELp2rqoLufNyFZR4gVzjnZc8V2VC3qEMEkJ6VrzeTTKgMMDjHDms0VLLwfh60mtg Sep 04 01:09:33.646141 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Id: None Sep 04 01:09:33.646141 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Name: None Sep 04 01:09:33.646141 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Identity-Status: Confirmed Sep 04 01:09:33.646141 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Is-Admin-Project: True Sep 04 01:09:33.646141 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Id: default Sep 04 01:09:33.646141 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Name: Default Sep 04 01:09:33.646141 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Id: 6071c3a795ba4aa098e3af8cde306d58 Sep 04 01:09:33.646141 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1378097567 Sep 04 01:09:33.646141 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Role: reader,member Sep 04 01:09:33.646141 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Roles: reader,member Sep 04 01:09:33.647813 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6071c3a795ba4aa098e3af8cde306d58"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_6071c3a795ba4aa098e3af8cde306d58", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/6071c3a795ba4aa098e3af8cde306d58"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/6071c3a795ba4aa098e3af8cde306d58"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:09:33.647813 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant: tempest-VolumesSnapshotTestJSON-1378097567 Sep 04 01:09:33.647813 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Id: 6071c3a795ba4aa098e3af8cde306d58 Sep 04 01:09:33.647813 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1378097567 Sep 04 01:09:33.647813 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User: tempest-VolumesSnapshotTestJSON-1378097567-project Sep 04 01:09:33.647813 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Id: default Sep 04 01:09:33.647813 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Name: Default Sep 04 01:09:33.647813 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Id: 77de7bb68f3c45d680c8a78dc40df07b Sep 04 01:09:33.647813 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Name: tempest-VolumesSnapshotTestJSON-1378097567-project Sep 04 01:09:33.647813 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: method: > Sep 04 01:09:33.647813 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: exception: Instance a3994b76-60e7-4d9a-a655-2fb09eb68a5d could not be found. Sep 04 01:09:33.647813 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: action_args: {'id': 'a3994b76-60e7-4d9a-a655-2fb09eb68a5d'} Sep 04 01:09:33.648816 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:09:33.648816 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:09:33.648816 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:09:33.648816 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:09:33.648816 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:09:33.648816 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:09:33.648816 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:09:33.648816 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:09:33.648816 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:09:33.648816 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:09:33.648816 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a3994b76-60e7-4d9a-a655-2fb09eb68a5d could not be found. Sep 04 01:09:33.648816 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:09:33.648816 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:09:33.648816 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:09:33.648816 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:09:33.648816 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:09:33.648816 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:09:33.648816 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:09:33.648816 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:09:33.648816 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:09:33.650070 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:09:33.650070 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 04 01:09:33.650070 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 04 01:09:33.650070 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:09:33.650070 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:09:33.650070 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:09:33.650070 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:09:33.650070 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a3994b76-60e7-4d9a-a655-2fb09eb68a5d could not be found. Sep 04 01:09:33.650070 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:09:33.650070 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:09:33.650070 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:09:33.650070 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:09:33.650070 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:09:33.650070 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:09:33.650070 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:09:33.650070 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:09:33.650070 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a3994b76-60e7-4d9a-a655-2fb09eb68a5d could not be found. Sep 04 01:09:33.650070 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:09:33.651114 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-e65d0be6-b28a-40f6-b559-2fd1b573a860 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] Returning 404 to user: Instance a3994b76-60e7-4d9a-a655-2fb09eb68a5d could not be found. {{(pid=121037) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:09:33.651114 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-e65d0be6-b28a-40f6-b559-2fd1b573a860 tempest-VolumesSnapshotTestJSON-1378097567 tempest-VolumesSnapshotTestJSON-1378097567-project] 10.222.0.42 "GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d" status: 404 len: 111 microversion: 2.1 time: 0.029868 Sep 04 01:09:33.651114 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 375/770] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:33 2022] GET /compute/v2.1/servers/a3994b76-60e7-4d9a-a655-2fb09eb68a5d => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 04 01:09:33.831905 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-91a9fe7c-5576-44a6-b7e0-9711d733120f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:33.832701 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-91a9fe7c-5576-44a6-b7e0-9711d733120f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:33.838955 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-91a9fe7c-5576-44a6-b7e0-9711d733120f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:33.839434 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-91a9fe7c-5576-44a6-b7e0-9711d733120f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:33.839953 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-91a9fe7c-5576-44a6-b7e0-9711d733120f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:33.840713 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-91a9fe7c-5576-44a6-b7e0-9711d733120f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:33.841255 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-91a9fe7c-5576-44a6-b7e0-9711d733120f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:33.841876 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-91a9fe7c-5576-44a6-b7e0-9711d733120f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:33.978083 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-91a9fe7c-5576-44a6-b7e0-9711d733120f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:33.986355 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-91a9fe7c-5576-44a6-b7e0-9711d733120f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1259 microversion: 2.1 time: 0.157964 Sep 04 01:09:33.990420 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 396/771] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:33 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1259 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:34.246109 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-edda8888-7798-476d-a26e-837eb31007f9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:34.246109 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-edda8888-7798-476d-a26e-837eb31007f9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:34.246109 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-edda8888-7798-476d-a26e-837eb31007f9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:34.246109 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-edda8888-7798-476d-a26e-837eb31007f9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:34.246492 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-edda8888-7798-476d-a26e-837eb31007f9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:34.246492 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-edda8888-7798-476d-a26e-837eb31007f9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:34.246492 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-edda8888-7798-476d-a26e-837eb31007f9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:34.246492 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-edda8888-7798-476d-a26e-837eb31007f9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:34.566025 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-edda8888-7798-476d-a26e-837eb31007f9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:34.570983 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-a8244930-ec45-419b-91e8-a3941a1270c1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:34.571668 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-a8244930-ec45-419b-91e8-a3941a1270c1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:34.574203 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-edda8888-7798-476d-a26e-837eb31007f9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1684 microversion: 2.1 time: 0.345060 Sep 04 01:09:34.575132 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 376/772] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:34 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1684 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:34.579234 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a8244930-ec45-419b-91e8-a3941a1270c1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:34.579804 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a8244930-ec45-419b-91e8-a3941a1270c1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:34.580330 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a8244930-ec45-419b-91e8-a3941a1270c1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:34.581127 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a8244930-ec45-419b-91e8-a3941a1270c1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:34.581653 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a8244930-ec45-419b-91e8-a3941a1270c1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:34.582322 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a8244930-ec45-419b-91e8-a3941a1270c1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:34.714819 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-a8244930-ec45-419b-91e8-a3941a1270c1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:34.723160 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-a8244930-ec45-419b-91e8-a3941a1270c1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1723 microversion: 2.1 time: 0.156312 Sep 04 01:09:34.732578 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 397/773] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:34 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1723 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:09:35.010270 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-47f52978-0c83-4e97-af5a-714d2259f1e4 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:35.010938 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-47f52978-0c83-4e97-af5a-714d2259f1e4 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:35.017762 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-47f52978-0c83-4e97-af5a-714d2259f1e4 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:35.018463 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-47f52978-0c83-4e97-af5a-714d2259f1e4 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:35.018911 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-47f52978-0c83-4e97-af5a-714d2259f1e4 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:35.019667 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-47f52978-0c83-4e97-af5a-714d2259f1e4 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:35.020192 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-47f52978-0c83-4e97-af5a-714d2259f1e4 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:35.020588 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-47f52978-0c83-4e97-af5a-714d2259f1e4 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:35.027470 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-7bb44524-801a-483c-bb24-435db62180d1 req-a02163af-d0cf-417e-8edf-1e28ca24bb46 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "784b2b9b-9890-4f9e-be75-2548b520e138", "name": "network-vif-deleted", "tag": "e0a049e7-02dc-4e07-af75-3de7fbe36b23"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:09:35.038293 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-7bb44524-801a-483c-bb24-435db62180d1 req-a02163af-d0cf-417e-8edf-1e28ca24bb46 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:35.039091 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-7bb44524-801a-483c-bb24-435db62180d1 req-a02163af-d0cf-417e-8edf-1e28ca24bb46 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:35.039524 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-7bb44524-801a-483c-bb24-435db62180d1 req-a02163af-d0cf-417e-8edf-1e28ca24bb46 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:35.051348 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-7bb44524-801a-483c-bb24-435db62180d1 req-a02163af-d0cf-417e-8edf-1e28ca24bb46 service nova] Creating event network-vif-deleted:e0a049e7-02dc-4e07-af75-3de7fbe36b23 for instance 784b2b9b-9890-4f9e-be75-2548b520e138 on hv-cinder-maste Sep 04 01:09:35.059049 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-7bb44524-801a-483c-bb24-435db62180d1 req-a02163af-d0cf-417e-8edf-1e28ca24bb46 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035228 Sep 04 01:09:35.059843 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 398/774] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:09:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:09:35.173740 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-47f52978-0c83-4e97-af5a-714d2259f1e4 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:35.182720 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-47f52978-0c83-4e97-af5a-714d2259f1e4 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1259 microversion: 2.1 time: 0.177048 Sep 04 01:09:35.197820 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 377/775] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:35 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1259 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:35.599371 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-48eda144-abdb-4e07-8bbc-832468480f2c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:35.600671 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-48eda144-abdb-4e07-8bbc-832468480f2c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:35.607361 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-48eda144-abdb-4e07-8bbc-832468480f2c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:35.608063 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-48eda144-abdb-4e07-8bbc-832468480f2c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:35.608867 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-48eda144-abdb-4e07-8bbc-832468480f2c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:35.609806 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-48eda144-abdb-4e07-8bbc-832468480f2c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:35.610471 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-48eda144-abdb-4e07-8bbc-832468480f2c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:35.611358 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-48eda144-abdb-4e07-8bbc-832468480f2c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:35.684456 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-48eda144-abdb-4e07-8bbc-832468480f2c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:35.692850 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-48eda144-abdb-4e07-8bbc-832468480f2c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1616 microversion: 2.1 time: 0.098231 Sep 04 01:09:35.693666 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 399/776] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:35 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1616 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:35.747177 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-8eabc60f-1c56-4aa5-9a8c-4427ce923e3d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:35.747781 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-8eabc60f-1c56-4aa5-9a8c-4427ce923e3d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:35.754253 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8eabc60f-1c56-4aa5-9a8c-4427ce923e3d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:35.754738 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8eabc60f-1c56-4aa5-9a8c-4427ce923e3d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:35.755173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8eabc60f-1c56-4aa5-9a8c-4427ce923e3d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:35.756701 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8eabc60f-1c56-4aa5-9a8c-4427ce923e3d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:35.757181 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8eabc60f-1c56-4aa5-9a8c-4427ce923e3d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:35.757610 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8eabc60f-1c56-4aa5-9a8c-4427ce923e3d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:35.897041 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-8eabc60f-1c56-4aa5-9a8c-4427ce923e3d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:35.908138 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-8eabc60f-1c56-4aa5-9a8c-4427ce923e3d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1723 microversion: 2.1 time: 0.164819 Sep 04 01:09:35.908718 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 378/777] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:35 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1723 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:36.206761 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-44bff795-b31b-489a-8627-5ee666400751 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:36.207364 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-44bff795-b31b-489a-8627-5ee666400751 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:36.213498 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-44bff795-b31b-489a-8627-5ee666400751 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:36.213977 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-44bff795-b31b-489a-8627-5ee666400751 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:36.214408 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-44bff795-b31b-489a-8627-5ee666400751 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:36.215125 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-44bff795-b31b-489a-8627-5ee666400751 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:36.215571 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-44bff795-b31b-489a-8627-5ee666400751 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:36.215988 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-44bff795-b31b-489a-8627-5ee666400751 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:36.344207 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-44bff795-b31b-489a-8627-5ee666400751 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:36.351829 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-44bff795-b31b-489a-8627-5ee666400751 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1259 microversion: 2.1 time: 0.148946 Sep 04 01:09:36.352390 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 400/778] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:36 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1259 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:36.715659 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-e04800a8-76ab-4884-8a13-80b738beafc8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:36.716262 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-e04800a8-76ab-4884-8a13-80b738beafc8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:36.722342 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e04800a8-76ab-4884-8a13-80b738beafc8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:36.722817 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e04800a8-76ab-4884-8a13-80b738beafc8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:36.723250 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e04800a8-76ab-4884-8a13-80b738beafc8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:36.723934 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e04800a8-76ab-4884-8a13-80b738beafc8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:36.724901 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e04800a8-76ab-4884-8a13-80b738beafc8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:36.725356 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e04800a8-76ab-4884-8a13-80b738beafc8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:36.789864 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-e04800a8-76ab-4884-8a13-80b738beafc8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:36.799842 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-e04800a8-76ab-4884-8a13-80b738beafc8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1616 microversion: 2.1 time: 0.087953 Sep 04 01:09:36.800414 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 379/779] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:36 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1616 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:36.931422 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-abae3fd8-313b-46c8-a603-5484cb89fda7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:36.931962 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-abae3fd8-313b-46c8-a603-5484cb89fda7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:36.938362 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-abae3fd8-313b-46c8-a603-5484cb89fda7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:36.938834 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-abae3fd8-313b-46c8-a603-5484cb89fda7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:36.939264 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-abae3fd8-313b-46c8-a603-5484cb89fda7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:36.939944 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-abae3fd8-313b-46c8-a603-5484cb89fda7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:36.940387 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-abae3fd8-313b-46c8-a603-5484cb89fda7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:36.940809 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-abae3fd8-313b-46c8-a603-5484cb89fda7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:37.067176 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-abae3fd8-313b-46c8-a603-5484cb89fda7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:37.075443 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-abae3fd8-313b-46c8-a603-5484cb89fda7 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1723 microversion: 2.1 time: 0.148012 Sep 04 01:09:37.076195 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 401/780] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:36 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1723 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:37.374607 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-a6421b03-31a6-4eb4-b340-8c0d4ecc4ead tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:37.375216 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-a6421b03-31a6-4eb4-b340-8c0d4ecc4ead tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:37.381659 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a6421b03-31a6-4eb4-b340-8c0d4ecc4ead tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:37.382131 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a6421b03-31a6-4eb4-b340-8c0d4ecc4ead tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:37.382562 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a6421b03-31a6-4eb4-b340-8c0d4ecc4ead tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:37.383243 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a6421b03-31a6-4eb4-b340-8c0d4ecc4ead tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:37.383691 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a6421b03-31a6-4eb4-b340-8c0d4ecc4ead tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:37.384114 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a6421b03-31a6-4eb4-b340-8c0d4ecc4ead tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:37.519358 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-a6421b03-31a6-4eb4-b340-8c0d4ecc4ead tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:37.528248 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-a6421b03-31a6-4eb4-b340-8c0d4ecc4ead tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1259 microversion: 2.1 time: 0.157518 Sep 04 01:09:37.528810 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 380/781] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:37 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1259 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:37.829927 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-7187bf1a-4239-4532-a144-3b7e48720c76 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:37.830563 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-7187bf1a-4239-4532-a144-3b7e48720c76 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:37.836639 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7187bf1a-4239-4532-a144-3b7e48720c76 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:37.837152 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7187bf1a-4239-4532-a144-3b7e48720c76 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:37.837589 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7187bf1a-4239-4532-a144-3b7e48720c76 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:37.838270 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7187bf1a-4239-4532-a144-3b7e48720c76 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:37.838718 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7187bf1a-4239-4532-a144-3b7e48720c76 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:37.839141 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7187bf1a-4239-4532-a144-3b7e48720c76 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:37.904267 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-7187bf1a-4239-4532-a144-3b7e48720c76 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:37.912034 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-7187bf1a-4239-4532-a144-3b7e48720c76 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1616 microversion: 2.1 time: 0.086034 Sep 04 01:09:37.912844 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 402/782] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:37 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1616 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:38.099128 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-cc7ca943-5513-44d9-acea-8bbd32534947 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:38.100335 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-cc7ca943-5513-44d9-acea-8bbd32534947 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:38.106860 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-cc7ca943-5513-44d9-acea-8bbd32534947 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:38.107335 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-cc7ca943-5513-44d9-acea-8bbd32534947 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:38.107765 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-cc7ca943-5513-44d9-acea-8bbd32534947 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:38.108443 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-cc7ca943-5513-44d9-acea-8bbd32534947 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:38.108885 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-cc7ca943-5513-44d9-acea-8bbd32534947 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:38.109409 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-cc7ca943-5513-44d9-acea-8bbd32534947 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:38.235500 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-cc7ca943-5513-44d9-acea-8bbd32534947 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:38.244501 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-cc7ca943-5513-44d9-acea-8bbd32534947 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1723 microversion: 2.1 time: 0.149197 Sep 04 01:09:38.245066 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 381/783] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:38 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1723 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:38.550996 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-93f0d490-30cc-4258-ba57-b1490609397a tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:38.551635 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-93f0d490-30cc-4258-ba57-b1490609397a tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:38.558072 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-93f0d490-30cc-4258-ba57-b1490609397a tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:38.558552 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-93f0d490-30cc-4258-ba57-b1490609397a tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:38.558989 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-93f0d490-30cc-4258-ba57-b1490609397a tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:38.559677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-93f0d490-30cc-4258-ba57-b1490609397a tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:38.560126 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-93f0d490-30cc-4258-ba57-b1490609397a tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:38.560550 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-93f0d490-30cc-4258-ba57-b1490609397a tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:38.564518 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=100 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:09:38.688055 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-93f0d490-30cc-4258-ba57-b1490609397a tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:38.695789 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-93f0d490-30cc-4258-ba57-b1490609397a tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1259 microversion: 2.1 time: 0.148677 Sep 04 01:09:38.696359 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 403/784] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:38 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1259 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:38.936496 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-35c54be5-5a7a-400e-a4d5-66b249862ddb tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:38.937028 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-35c54be5-5a7a-400e-a4d5-66b249862ddb tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:38.943740 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-35c54be5-5a7a-400e-a4d5-66b249862ddb tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:38.944270 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-35c54be5-5a7a-400e-a4d5-66b249862ddb tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:38.944702 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-35c54be5-5a7a-400e-a4d5-66b249862ddb tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:38.945429 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-35c54be5-5a7a-400e-a4d5-66b249862ddb tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:38.945875 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-35c54be5-5a7a-400e-a4d5-66b249862ddb tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:38.946292 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-35c54be5-5a7a-400e-a4d5-66b249862ddb tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:39.014477 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-35c54be5-5a7a-400e-a4d5-66b249862ddb tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:39.022981 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-35c54be5-5a7a-400e-a4d5-66b249862ddb tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1616 microversion: 2.1 time: 0.090986 Sep 04 01:09:39.023539 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 382/785] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:38 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1616 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:39.267736 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-e75ccf09-be59-46a5-88a9-fa9a6c3b3777 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:39.268266 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-e75ccf09-be59-46a5-88a9-fa9a6c3b3777 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:39.274352 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e75ccf09-be59-46a5-88a9-fa9a6c3b3777 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:39.274834 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e75ccf09-be59-46a5-88a9-fa9a6c3b3777 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:39.275266 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e75ccf09-be59-46a5-88a9-fa9a6c3b3777 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:39.275957 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e75ccf09-be59-46a5-88a9-fa9a6c3b3777 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:39.276410 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e75ccf09-be59-46a5-88a9-fa9a6c3b3777 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:39.276835 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e75ccf09-be59-46a5-88a9-fa9a6c3b3777 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:39.403186 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e75ccf09-be59-46a5-88a9-fa9a6c3b3777 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:39.411729 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-e75ccf09-be59-46a5-88a9-fa9a6c3b3777 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1723 microversion: 2.1 time: 0.147930 Sep 04 01:09:39.412705 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 404/786] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:39 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1723 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:39.718997 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-27af4342-882d-4c5d-9575-914565f21155 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:39.719622 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-27af4342-882d-4c5d-9575-914565f21155 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:39.726663 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-27af4342-882d-4c5d-9575-914565f21155 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:39.727134 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-27af4342-882d-4c5d-9575-914565f21155 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:39.727562 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-27af4342-882d-4c5d-9575-914565f21155 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:39.728242 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-27af4342-882d-4c5d-9575-914565f21155 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:39.728687 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-27af4342-882d-4c5d-9575-914565f21155 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:39.729164 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-27af4342-882d-4c5d-9575-914565f21155 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:39.731781 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:09:39.876753 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-27af4342-882d-4c5d-9575-914565f21155 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:39.885806 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-27af4342-882d-4c5d-9575-914565f21155 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1259 microversion: 2.1 time: 0.170632 Sep 04 01:09:39.886384 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 383/787] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:39 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1259 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:40.046402 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-ad6277e9-5b47-48a6-9a8c-2230236235d5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:40.047011 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-ad6277e9-5b47-48a6-9a8c-2230236235d5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:40.053568 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ad6277e9-5b47-48a6-9a8c-2230236235d5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:40.054046 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ad6277e9-5b47-48a6-9a8c-2230236235d5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:40.054478 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ad6277e9-5b47-48a6-9a8c-2230236235d5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:40.055165 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ad6277e9-5b47-48a6-9a8c-2230236235d5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:40.055617 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ad6277e9-5b47-48a6-9a8c-2230236235d5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:40.056038 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ad6277e9-5b47-48a6-9a8c-2230236235d5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:40.087596 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ad6277e9-5b47-48a6-9a8c-2230236235d5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:40.088073 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ad6277e9-5b47-48a6-9a8c-2230236235d5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:40.088510 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ad6277e9-5b47-48a6-9a8c-2230236235d5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:40.088953 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.objects.instance [None req-ad6277e9-5b47-48a6-9a8c-2230236235d5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lazy-loading 'fault' on Instance uuid 784b2b9b-9890-4f9e-be75-2548b520e138 {{(pid=121036) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:09:40.132611 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ad6277e9-5b47-48a6-9a8c-2230236235d5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:40.140449 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-ad6277e9-5b47-48a6-9a8c-2230236235d5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1332 microversion: 2.1 time: 0.097896 Sep 04 01:09:40.141015 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 405/788] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:40 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1332 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:40.438401 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-c55e5d12-6cea-4572-888a-2e56dc9d751e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:40.439024 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-c55e5d12-6cea-4572-888a-2e56dc9d751e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:40.446002 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c55e5d12-6cea-4572-888a-2e56dc9d751e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:40.446491 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c55e5d12-6cea-4572-888a-2e56dc9d751e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:40.447087 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c55e5d12-6cea-4572-888a-2e56dc9d751e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:40.447796 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c55e5d12-6cea-4572-888a-2e56dc9d751e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:40.448250 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c55e5d12-6cea-4572-888a-2e56dc9d751e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:40.448681 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c55e5d12-6cea-4572-888a-2e56dc9d751e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:40.585318 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-c55e5d12-6cea-4572-888a-2e56dc9d751e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:40.594168 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-c55e5d12-6cea-4572-888a-2e56dc9d751e tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1729 microversion: 2.1 time: 0.160998 Sep 04 01:09:40.594732 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 384/789] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:40 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1729 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:40.618132 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-782d5728-f0be-4b1d-82ff-a710dbb7d000 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:40.618623 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-782d5728-f0be-4b1d-82ff-a710dbb7d000 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:40.625945 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-782d5728-f0be-4b1d-82ff-a710dbb7d000 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:40.627841 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-782d5728-f0be-4b1d-82ff-a710dbb7d000 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:40.627841 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-782d5728-f0be-4b1d-82ff-a710dbb7d000 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:40.627841 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-782d5728-f0be-4b1d-82ff-a710dbb7d000 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:40.628242 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-782d5728-f0be-4b1d-82ff-a710dbb7d000 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:40.628680 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-782d5728-f0be-4b1d-82ff-a710dbb7d000 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:40.708236 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-782d5728-f0be-4b1d-82ff-a710dbb7d000 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.094623 Sep 04 01:09:40.708893 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 406/790] 10.222.0.42 () {58 vars in 1275 bytes} [Sun Sep 4 01:09:40 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134/os-interface => generated 284 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:09:40.911003 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-fbd10d5a-9159-44c5-b9dd-ed56eff1a06c tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:40.911636 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-fbd10d5a-9159-44c5-b9dd-ed56eff1a06c tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:40.920420 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fbd10d5a-9159-44c5-b9dd-ed56eff1a06c tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:40.921167 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fbd10d5a-9159-44c5-b9dd-ed56eff1a06c tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:40.925122 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fbd10d5a-9159-44c5-b9dd-ed56eff1a06c tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:40.925122 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fbd10d5a-9159-44c5-b9dd-ed56eff1a06c tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:40.925122 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fbd10d5a-9159-44c5-b9dd-ed56eff1a06c tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:40.925122 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fbd10d5a-9159-44c5-b9dd-ed56eff1a06c tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:41.076130 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-fbd10d5a-9159-44c5-b9dd-ed56eff1a06c tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:41.089817 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-fbd10d5a-9159-44c5-b9dd-ed56eff1a06c tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1259 microversion: 2.1 time: 0.183852 Sep 04 01:09:41.090597 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 385/791] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:40 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1259 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:41.168665 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-00bd917c-9b97-4984-ab91-3ad2c7afdfaf tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:41.169392 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-00bd917c-9b97-4984-ab91-3ad2c7afdfaf tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:41.175611 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-00bd917c-9b97-4984-ab91-3ad2c7afdfaf tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:41.176168 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-00bd917c-9b97-4984-ab91-3ad2c7afdfaf tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:41.176613 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-00bd917c-9b97-4984-ab91-3ad2c7afdfaf tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:41.177335 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-00bd917c-9b97-4984-ab91-3ad2c7afdfaf tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:41.178497 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-00bd917c-9b97-4984-ab91-3ad2c7afdfaf tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:41.179027 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-00bd917c-9b97-4984-ab91-3ad2c7afdfaf tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:41.212246 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-00bd917c-9b97-4984-ab91-3ad2c7afdfaf tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:41.212735 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-00bd917c-9b97-4984-ab91-3ad2c7afdfaf tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:41.213189 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-00bd917c-9b97-4984-ab91-3ad2c7afdfaf tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:41.213641 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.objects.instance [None req-00bd917c-9b97-4984-ab91-3ad2c7afdfaf tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lazy-loading 'fault' on Instance uuid 784b2b9b-9890-4f9e-be75-2548b520e138 {{(pid=121036) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:09:41.285047 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-00bd917c-9b97-4984-ab91-3ad2c7afdfaf tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:41.293628 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-00bd917c-9b97-4984-ab91-3ad2c7afdfaf tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 200 len: 1332 microversion: 2.1 time: 0.129502 Sep 04 01:09:41.294200 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 407/792] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:41 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 1332 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:41.796328 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-76a923c3-0535-4c7e-84d8-252352a411f3 req-f54dd38d-4fd3-414e-b51c-16db8db312fa service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2570ee6c-f335-41bc-8e67-f89a62b28134", "tag": "62f5079a-1d7d-4c5c-bdcd-0af337b6235b"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:09:41.809999 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-76a923c3-0535-4c7e-84d8-252352a411f3 req-f54dd38d-4fd3-414e-b51c-16db8db312fa service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:41.810568 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-76a923c3-0535-4c7e-84d8-252352a411f3 req-f54dd38d-4fd3-414e-b51c-16db8db312fa service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:41.811125 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-76a923c3-0535-4c7e-84d8-252352a411f3 req-f54dd38d-4fd3-414e-b51c-16db8db312fa service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:41.822924 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-76a923c3-0535-4c7e-84d8-252352a411f3 req-f54dd38d-4fd3-414e-b51c-16db8db312fa service nova] Creating event network-changed:62f5079a-1d7d-4c5c-bdcd-0af337b6235b for instance 2570ee6c-f335-41bc-8e67-f89a62b28134 on hv-cinder-maste Sep 04 01:09:41.828218 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-76a923c3-0535-4c7e-84d8-252352a411f3 req-f54dd38d-4fd3-414e-b51c-16db8db312fa service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036976 Sep 04 01:09:41.829036 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 386/793] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:09:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:09:42.120667 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-9fd6c17e-c9e9-4d0c-808b-5ed635b2af65 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:42.121376 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-9fd6c17e-c9e9-4d0c-808b-5ed635b2af65 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:42.128072 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9fd6c17e-c9e9-4d0c-808b-5ed635b2af65 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:42.134998 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9fd6c17e-c9e9-4d0c-808b-5ed635b2af65 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:42.134998 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9fd6c17e-c9e9-4d0c-808b-5ed635b2af65 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:42.134998 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9fd6c17e-c9e9-4d0c-808b-5ed635b2af65 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:42.134998 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9fd6c17e-c9e9-4d0c-808b-5ed635b2af65 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:42.135404 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9fd6c17e-c9e9-4d0c-808b-5ed635b2af65 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:42.283025 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-9fd6c17e-c9e9-4d0c-808b-5ed635b2af65 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:42.293777 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-9fd6c17e-c9e9-4d0c-808b-5ed635b2af65 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1259 microversion: 2.1 time: 0.176991 Sep 04 01:09:42.295046 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 408/794] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:42 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1259 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:42.318851 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-5bd90639-8dc8-4c9f-ac62-e507629c3942 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:42.319843 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-5bd90639-8dc8-4c9f-ac62-e507629c3942 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:42.326172 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5bd90639-8dc8-4c9f-ac62-e507629c3942 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:42.326653 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5bd90639-8dc8-4c9f-ac62-e507629c3942 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:42.327084 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5bd90639-8dc8-4c9f-ac62-e507629c3942 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:42.327776 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5bd90639-8dc8-4c9f-ac62-e507629c3942 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:42.328220 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5bd90639-8dc8-4c9f-ac62-e507629c3942 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:42.328639 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5bd90639-8dc8-4c9f-ac62-e507629c3942 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:42.345404 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.wsgi [None req-5bd90639-8dc8-4c9f-ac62-e507629c3942 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] HTTP exception thrown: Instance 784b2b9b-9890-4f9e-be75-2548b520e138 could not be found. Sep 04 01:09:42.347350 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-5bd90639-8dc8-4c9f-ac62-e507629c3942 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Request method failure captured: Sep 04 01:09:42.347350 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: request: GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 HTTP/1.1 Sep 04 01:09:42.347350 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept: application/json Sep 04 01:09:42.347350 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept-Encoding: identity Sep 04 01:09:42.347350 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Connection: close Sep 04 01:09:42.347350 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Length: 0 Sep 04 01:09:42.347350 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Type: application/json Sep 04 01:09:42.347350 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Host: 10.222.0.42 Sep 04 01:09:42.347350 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Openstack-System-Scope: None Sep 04 01:09:42.347350 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: User-Agent: python-urllib3/1.26.12 Sep 04 01:09:42.347350 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Auth-Token: gAAAAABjE_nLdulKsCh7rP69EyY8StqBiRnQNFlCX5VoWZ9XzP_YTEis0NNuYGIELGjVYA3FFaOCC9PLYuhPddV3AgDhoAQw_S2o9MK0q1A57ehVT3gYS6mSkklOOLScx3a8r0TUFxeOMhMymWFJt03FSLp4K4nMUe8FttsVqGRsky_ik39T3Zg Sep 04 01:09:42.347350 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Id: None Sep 04 01:09:42.347350 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Name: None Sep 04 01:09:42.347350 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Identity-Status: Confirmed Sep 04 01:09:42.347350 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Is-Admin-Project: True Sep 04 01:09:42.347350 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Id: default Sep 04 01:09:42.347350 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Name: Default Sep 04 01:09:42.347350 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Id: 287d111f08ac4b05bd5adec9c40d0620 Sep 04 01:09:42.347350 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Name: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:09:42.347350 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Role: reader,member Sep 04 01:09:42.347350 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Roles: reader,member Sep 04 01:09:42.349057 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_287d111f08ac4b05bd5adec9c40d0620", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:09:42.349057 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:09:42.349057 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Id: 287d111f08ac4b05bd5adec9c40d0620 Sep 04 01:09:42.349057 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:09:42.349057 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User: tempest-AttachVolumeNegativeTest-825474236-project Sep 04 01:09:42.349057 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Id: default Sep 04 01:09:42.349057 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Name: Default Sep 04 01:09:42.349057 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Id: f5e4723258c64ce5885154dded2d3d43 Sep 04 01:09:42.349057 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Name: tempest-AttachVolumeNegativeTest-825474236-project Sep 04 01:09:42.349057 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: method: > Sep 04 01:09:42.349057 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: exception: Instance 784b2b9b-9890-4f9e-be75-2548b520e138 could not be found. Sep 04 01:09:42.349057 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: action_args: {'id': '784b2b9b-9890-4f9e-be75-2548b520e138'} Sep 04 01:09:42.350144 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:09:42.350144 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:09:42.350144 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:09:42.350144 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:09:42.350144 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:09:42.350144 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:09:42.350144 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:09:42.350144 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:09:42.350144 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:09:42.350144 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:09:42.350144 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 784b2b9b-9890-4f9e-be75-2548b520e138 could not be found. Sep 04 01:09:42.350144 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:09:42.350144 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:09:42.350144 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:09:42.350144 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:09:42.350144 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:09:42.350144 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:09:42.350144 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:09:42.350144 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:09:42.350144 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:09:42.351504 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:09:42.351504 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 04 01:09:42.351504 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 04 01:09:42.351504 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:09:42.351504 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:09:42.351504 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:09:42.351504 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:09:42.351504 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 784b2b9b-9890-4f9e-be75-2548b520e138 could not be found. Sep 04 01:09:42.351504 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:09:42.351504 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:09:42.351504 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:09:42.351504 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:09:42.351504 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:09:42.351504 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:09:42.351504 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:09:42.351504 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:09:42.351504 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 784b2b9b-9890-4f9e-be75-2548b520e138 could not be found. Sep 04 01:09:42.351504 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:09:42.354080 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-5bd90639-8dc8-4c9f-ac62-e507629c3942 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Returning 404 to user: Instance 784b2b9b-9890-4f9e-be75-2548b520e138 could not be found. {{(pid=121037) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:09:42.354961 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-5bd90639-8dc8-4c9f-ac62-e507629c3942 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 404 len: 111 microversion: 2.1 time: 0.041050 Sep 04 01:09:42.358944 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 387/795] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:42 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 04 01:09:42.364868 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-9e230a46-d80e-4c67-b953-108852768a42 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '.version_select at 0x7fc10c4f9c10>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:42.463454 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-9e230a46-d80e-4c67-b953-108852768a42 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1366067967" status: 202 len: 0 microversion: 2.1 time: 0.097850 Sep 04 01:09:42.464339 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 409/796] 10.222.0.42 () {58 vars in 1221 bytes} [Sun Sep 4 01:09:42 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1366067967 => generated 0 bytes in 104 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 04 01:09:42.988478 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-8e6a6421-ee0f-44ac-b3c3-b1ff5db4779b req-d8ad1a90-26d2-4dd2-bcbe-e835d8d53fc5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "74df11dc-31bd-4745-83bc-4b3c430d02ff", "name": "network-vif-plugged", "status": "completed", "tag": "d072bfcf-41bd-430e-800c-1d1a3b17d915"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:09:42.995176 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_api:SELECT=98,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:09:43.000480 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-8e6a6421-ee0f-44ac-b3c3-b1ff5db4779b req-d8ad1a90-26d2-4dd2-bcbe-e835d8d53fc5 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:43.000903 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-8e6a6421-ee0f-44ac-b3c3-b1ff5db4779b req-d8ad1a90-26d2-4dd2-bcbe-e835d8d53fc5 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:43.001363 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-8e6a6421-ee0f-44ac-b3c3-b1ff5db4779b req-d8ad1a90-26d2-4dd2-bcbe-e835d8d53fc5 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:43.013047 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-8e6a6421-ee0f-44ac-b3c3-b1ff5db4779b req-d8ad1a90-26d2-4dd2-bcbe-e835d8d53fc5 service nova] Creating event network-vif-plugged:d072bfcf-41bd-430e-800c-1d1a3b17d915 for instance 74df11dc-31bd-4745-83bc-4b3c430d02ff on hv-cinder-maste Sep 04 01:09:43.017924 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-8e6a6421-ee0f-44ac-b3c3-b1ff5db4779b req-d8ad1a90-26d2-4dd2-bcbe-e835d8d53fc5 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033975 Sep 04 01:09:43.018701 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 388/797] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:09:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:09:43.257065 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b2e1f580-15c6-431a-be88-0b9cf9fcc2c2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Action: 'create', calling method: .version_select at 0x7fc10c580550>, body: {"keypair": {"name": "tempest-keypair-510224483", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFkDCqYzlI/RIxL7qjwuKDBt7i77QyW9K60rNpJAW+VEmUnn+OHhDw5FMgjX7ivu+k3D1JqdJk8N9XZ2XPOgCcmZfZ4Iyg5OcrvRe33OpGVR3UHOMaOHuwMI2cciJyVtLg=="}} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:09:43.269201 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-b2e1f580-15c6-431a-be88-0b9cf9fcc2c2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Getting quotas for project 287d111f08ac4b05bd5adec9c40d0620. Resources: {'key_pairs'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 04 01:09:43.273266 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-b2e1f580-15c6-431a-be88-0b9cf9fcc2c2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Getting quotas for user f5e4723258c64ce5885154dded2d3d43 and project 287d111f08ac4b05bd5adec9c40d0620. Resources: {'key_pairs'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 04 01:09:43.317779 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-76259d8a-9846-4cae-81f7-ce1f6ba12571 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:43.319264 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-76259d8a-9846-4cae-81f7-ce1f6ba12571 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:43.326203 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-76259d8a-9846-4cae-81f7-ce1f6ba12571 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:43.326683 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-76259d8a-9846-4cae-81f7-ce1f6ba12571 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:43.327113 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-76259d8a-9846-4cae-81f7-ce1f6ba12571 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:43.327797 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-76259d8a-9846-4cae-81f7-ce1f6ba12571 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:43.328242 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-76259d8a-9846-4cae-81f7-ce1f6ba12571 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:43.328660 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-76259d8a-9846-4cae-81f7-ce1f6ba12571 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:43.374309 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-b2e1f580-15c6-431a-be88-0b9cf9fcc2c2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.119263 Sep 04 01:09:43.374309 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 410/798] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Sep 4 01:09:43 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:09:43.530149 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-76259d8a-9846-4cae-81f7-ce1f6ba12571 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:43.539713 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-76259d8a-9846-4cae-81f7-ce1f6ba12571 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1259 microversion: 2.1 time: 0.226433 Sep 04 01:09:43.540567 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 389/799] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:43 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1259 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:09:43.845110 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-7959bfcb-870f-4246-b70d-fb786ab931b1 req-8cc2fef1-416b-409e-93c8-d4639674b7fe service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2570ee6c-f335-41bc-8e67-f89a62b28134", "tag": "62f5079a-1d7d-4c5c-bdcd-0af337b6235b"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:09:43.855110 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-7959bfcb-870f-4246-b70d-fb786ab931b1 req-8cc2fef1-416b-409e-93c8-d4639674b7fe service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:43.855582 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-7959bfcb-870f-4246-b70d-fb786ab931b1 req-8cc2fef1-416b-409e-93c8-d4639674b7fe service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:43.856009 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-7959bfcb-870f-4246-b70d-fb786ab931b1 req-8cc2fef1-416b-409e-93c8-d4639674b7fe service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:43.867146 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-7959bfcb-870f-4246-b70d-fb786ab931b1 req-8cc2fef1-416b-409e-93c8-d4639674b7fe service nova] Creating event network-changed:62f5079a-1d7d-4c5c-bdcd-0af337b6235b for instance 2570ee6c-f335-41bc-8e67-f89a62b28134 on hv-cinder-maste Sep 04 01:09:43.872197 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-7959bfcb-870f-4246-b70d-fb786ab931b1 req-8cc2fef1-416b-409e-93c8-d4639674b7fe service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030988 Sep 04 01:09:43.872781 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 411/800] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:09:43 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:09:44.563428 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-ed44fc05-0fe7-49cf-be80-e53da7e92ebc tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:44.564112 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-ed44fc05-0fe7-49cf-be80-e53da7e92ebc tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:44.571442 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ed44fc05-0fe7-49cf-be80-e53da7e92ebc tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:44.571992 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ed44fc05-0fe7-49cf-be80-e53da7e92ebc tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:44.572424 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ed44fc05-0fe7-49cf-be80-e53da7e92ebc tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:44.573127 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ed44fc05-0fe7-49cf-be80-e53da7e92ebc tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:44.573581 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ed44fc05-0fe7-49cf-be80-e53da7e92ebc tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:44.574158 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ed44fc05-0fe7-49cf-be80-e53da7e92ebc tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:45.047904 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-ed44fc05-0fe7-49cf-be80-e53da7e92ebc tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:45.055760 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-ed44fc05-0fe7-49cf-be80-e53da7e92ebc tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1259 microversion: 2.1 time: 0.496250 Sep 04 01:09:45.056332 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 390/801] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:44 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1259 bytes in 497 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:45.918893 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-5797d1d4-3a5d-43d1-9de9-71a7ffa29c8d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-2110533564", "imageRef": "a11315c3-ce81-478b-91e4-5353f189dfa7", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "d7f39628-666e-4480-8bee-49de183d5e09"}], "security_groups": [{"name": "tempest-securitygroup--1895800531"}], "key_name": "tempest-keypair-510224483", "user_data": "}} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:09:46.025734 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.network.neutron [None req-5797d1d4-3a5d-43d1-9de9-71a7ffa29c8d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] validate_networks() for [('d7f39628-666e-4480-8bee-49de183d5e09', None, None, None, None, None)] {{(pid=121036) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 04 01:09:46.078829 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-67aee94c-36ec-4b0e-bda7-649c6d478ee0 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:46.079433 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-67aee94c-36ec-4b0e-bda7-649c6d478ee0 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:46.085966 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-67aee94c-36ec-4b0e-bda7-649c6d478ee0 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:46.090820 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-67aee94c-36ec-4b0e-bda7-649c6d478ee0 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:46.090820 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-67aee94c-36ec-4b0e-bda7-649c6d478ee0 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:46.090820 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-67aee94c-36ec-4b0e-bda7-649c6d478ee0 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:46.090820 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-67aee94c-36ec-4b0e-bda7-649c6d478ee0 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:46.091361 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-67aee94c-36ec-4b0e-bda7-649c6d478ee0 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:46.222051 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-67aee94c-36ec-4b0e-bda7-649c6d478ee0 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:46.230202 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-67aee94c-36ec-4b0e-bda7-649c6d478ee0 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1443 microversion: 2.1 time: 0.155198 Sep 04 01:09:46.230782 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 391/802] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:46 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1443 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:09:46.254189 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-66227601-5df1-4ab8-8fb0-1d393176a32a tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:46.254796 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-66227601-5df1-4ab8-8fb0-1d393176a32a tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:46.261546 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-66227601-5df1-4ab8-8fb0-1d393176a32a tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:46.262029 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-66227601-5df1-4ab8-8fb0-1d393176a32a tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:46.262462 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-66227601-5df1-4ab8-8fb0-1d393176a32a tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:46.263144 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-66227601-5df1-4ab8-8fb0-1d393176a32a tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:46.263595 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-66227601-5df1-4ab8-8fb0-1d393176a32a tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:46.264016 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-66227601-5df1-4ab8-8fb0-1d393176a32a tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:46.398074 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-66227601-5df1-4ab8-8fb0-1d393176a32a tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:46.404789 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-5797d1d4-3a5d-43d1-9de9-71a7ffa29c8d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Flavor limits 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 04 01:09:46.405348 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-5797d1d4-3a5d-43d1-9de9-71a7ffa29c8d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Image limits 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 04 01:09:46.405744 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-66227601-5df1-4ab8-8fb0-1d393176a32a tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1443 microversion: 2.1 time: 0.155385 Sep 04 01:09:46.405955 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-5797d1d4-3a5d-43d1-9de9-71a7ffa29c8d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Flavor pref 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 04 01:09:46.406310 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 392/803] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:46 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1443 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:09:46.406425 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-5797d1d4-3a5d-43d1-9de9-71a7ffa29c8d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Image pref 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 04 01:09:46.406835 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-5797d1d4-3a5d-43d1-9de9-71a7ffa29c8d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 04 01:09:46.409960 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_api:SELECT=97,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=1 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:09:46.416615 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5797d1d4-3a5d-43d1-9de9-71a7ffa29c8d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:46.419759 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5797d1d4-3a5d-43d1-9de9-71a7ffa29c8d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:46.419759 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5797d1d4-3a5d-43d1-9de9-71a7ffa29c8d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:46.419759 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5797d1d4-3a5d-43d1-9de9-71a7ffa29c8d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:46.419759 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5797d1d4-3a5d-43d1-9de9-71a7ffa29c8d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:46.420191 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5797d1d4-3a5d-43d1-9de9-71a7ffa29c8d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:46.437932 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-5797d1d4-3a5d-43d1-9de9-71a7ffa29c8d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Getting quotas for project 287d111f08ac4b05bd5adec9c40d0620. Resources: {'ram', 'instances', 'cores'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 04 01:09:46.441832 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-5797d1d4-3a5d-43d1-9de9-71a7ffa29c8d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Getting quotas for user f5e4723258c64ce5885154dded2d3d43 and project 287d111f08ac4b05bd5adec9c40d0620. Resources: {'ram', 'instances', 'cores'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 04 01:09:46.452883 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-5797d1d4-3a5d-43d1-9de9-71a7ffa29c8d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Going to run 1 instances... {{(pid=121036) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 04 01:09:46.455326 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-5797d1d4-3a5d-43d1-9de9-71a7ffa29c8d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 16b0498f-4cb8-491a-92cf-dac9890c7b4a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a11315c3-ce81-478b-91e4-5353f189dfa7',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121036) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 04 01:09:46.583193 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-5797d1d4-3a5d-43d1-9de9-71a7ffa29c8d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.669499 Sep 04 01:09:46.584039 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 412/804] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 4 01:09:45 2022] POST /compute/v2.1/servers => generated 402 bytes in 671 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 04 01:09:46.595219 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-82f3aaec-4978-45ba-af69-d0b6f3e04392 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:46.595822 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-82f3aaec-4978-45ba-af69-d0b6f3e04392 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 16b0498f-4cb8-491a-92cf-dac9890c7b4a] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:46.668368 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-82f3aaec-4978-45ba-af69-d0b6f3e04392 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:46.675227 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-82f3aaec-4978-45ba-af69-d0b6f3e04392 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a" status: 200 len: 1241 microversion: 2.1 time: 0.083890 Sep 04 01:09:46.675800 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 393/805] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:46 2022] GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a => generated 1241 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:47.698205 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-4b79cbc3-6803-4e48-afee-0b32d50fe395 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:47.698919 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-4b79cbc3-6803-4e48-afee-0b32d50fe395 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 16b0498f-4cb8-491a-92cf-dac9890c7b4a] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:47.706093 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4b79cbc3-6803-4e48-afee-0b32d50fe395 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:47.707832 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4b79cbc3-6803-4e48-afee-0b32d50fe395 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:47.707832 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4b79cbc3-6803-4e48-afee-0b32d50fe395 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:47.707832 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4b79cbc3-6803-4e48-afee-0b32d50fe395 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:47.708184 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4b79cbc3-6803-4e48-afee-0b32d50fe395 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:47.708605 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4b79cbc3-6803-4e48-afee-0b32d50fe395 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:47.770866 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-4b79cbc3-6803-4e48-afee-0b32d50fe395 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:47.779575 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-4b79cbc3-6803-4e48-afee-0b32d50fe395 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a" status: 200 len: 1245 microversion: 2.1 time: 0.085218 Sep 04 01:09:47.780167 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 413/806] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:47 2022] GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a => generated 1245 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:48.387337 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-e58153f5-3cd0-4db0-a6f9-ccdc7d802ca7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:48.387337 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-e58153f5-3cd0-4db0-a6f9-ccdc7d802ca7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:48.391458 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e58153f5-3cd0-4db0-a6f9-ccdc7d802ca7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:48.391982 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e58153f5-3cd0-4db0-a6f9-ccdc7d802ca7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:48.392423 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e58153f5-3cd0-4db0-a6f9-ccdc7d802ca7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:48.393147 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e58153f5-3cd0-4db0-a6f9-ccdc7d802ca7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:48.393597 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e58153f5-3cd0-4db0-a6f9-ccdc7d802ca7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:48.394185 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e58153f5-3cd0-4db0-a6f9-ccdc7d802ca7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:48.425804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-e58153f5-3cd0-4db0-a6f9-ccdc7d802ca7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3/os-volume_attachments/0108bcd2-226d-4293-a661-c18cff40bf1a" status: 200 len: 194 microversion: 2.20 time: 0.049910 Sep 04 01:09:48.426424 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 394/807] 10.222.0.42 () {60 vars in 1454 bytes} [Sun Sep 4 01:09:48 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3/os-volume_attachments/0108bcd2-226d-4293-a661-c18cff40bf1a => generated 194 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 04 01:09:48.512884 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-c561d167-c0b8-4c91-bd26-64eff29f0cfd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:48.513520 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-c561d167-c0b8-4c91-bd26-64eff29f0cfd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:48.519730 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c561d167-c0b8-4c91-bd26-64eff29f0cfd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:48.520235 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c561d167-c0b8-4c91-bd26-64eff29f0cfd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:48.520674 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c561d167-c0b8-4c91-bd26-64eff29f0cfd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:48.521419 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c561d167-c0b8-4c91-bd26-64eff29f0cfd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:48.521868 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c561d167-c0b8-4c91-bd26-64eff29f0cfd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:48.522288 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c561d167-c0b8-4c91-bd26-64eff29f0cfd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:48.612256 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.objects.instance [None req-c561d167-c0b8-4c91-bd26-64eff29f0cfd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lazy-loading 'services' on Instance uuid 49be5a0b-f748-486e-a546-c07a8e8451a3 {{(pid=121036) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:09:48.658412 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-20097391-e114-4ddd-85bc-397629d3d6df req-5b0c9d9e-7acc-4630-9d73-ce1fa986c592 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "74df11dc-31bd-4745-83bc-4b3c430d02ff", "tag": "d072bfcf-41bd-430e-800c-1d1a3b17d915"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:09:48.672373 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-20097391-e114-4ddd-85bc-397629d3d6df req-5b0c9d9e-7acc-4630-9d73-ce1fa986c592 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:48.672852 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-20097391-e114-4ddd-85bc-397629d3d6df req-5b0c9d9e-7acc-4630-9d73-ce1fa986c592 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:48.673305 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-20097391-e114-4ddd-85bc-397629d3d6df req-5b0c9d9e-7acc-4630-9d73-ce1fa986c592 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:48.688769 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-20097391-e114-4ddd-85bc-397629d3d6df req-5b0c9d9e-7acc-4630-9d73-ce1fa986c592 service nova] Creating event network-changed:d072bfcf-41bd-430e-800c-1d1a3b17d915 for instance 74df11dc-31bd-4745-83bc-4b3c430d02ff on hv-cinder-maste Sep 04 01:09:48.696752 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-20097391-e114-4ddd-85bc-397629d3d6df req-5b0c9d9e-7acc-4630-9d73-ce1fa986c592 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042565 Sep 04 01:09:48.700433 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 395/808] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:09:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:09:48.804225 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-18fcf2d8-08e6-4b74-8656-daacf952bc94 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:48.806100 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-18fcf2d8-08e6-4b74-8656-daacf952bc94 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 16b0498f-4cb8-491a-92cf-dac9890c7b4a] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:48.821269 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-18fcf2d8-08e6-4b74-8656-daacf952bc94 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:48.821760 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-18fcf2d8-08e6-4b74-8656-daacf952bc94 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:48.822204 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-18fcf2d8-08e6-4b74-8656-daacf952bc94 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:48.822927 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-18fcf2d8-08e6-4b74-8656-daacf952bc94 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:48.823379 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-18fcf2d8-08e6-4b74-8656-daacf952bc94 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:48.823806 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-18fcf2d8-08e6-4b74-8656-daacf952bc94 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:48.851728 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-c561d167-c0b8-4c91-bd26-64eff29f0cfd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "DELETE /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3/os-volume_attachments/0108bcd2-226d-4293-a661-c18cff40bf1a" status: 202 len: 0 microversion: 2.20 time: 0.342835 Sep 04 01:09:48.854482 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 414/809] 10.222.0.42 () {60 vars in 1457 bytes} [Sun Sep 4 01:09:48 2022] DELETE /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3/os-volume_attachments/0108bcd2-226d-4293-a661-c18cff40bf1a => generated 0 bytes in 346 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Sep 04 01:09:48.865908 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-f0a3e9cb-0fea-46a1-998a-7a103358d042 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:48.870221 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-f0a3e9cb-0fea-46a1-998a-7a103358d042 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:48.876275 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f0a3e9cb-0fea-46a1-998a-7a103358d042 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:48.877013 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f0a3e9cb-0fea-46a1-998a-7a103358d042 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:48.877743 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f0a3e9cb-0fea-46a1-998a-7a103358d042 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:48.879762 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f0a3e9cb-0fea-46a1-998a-7a103358d042 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:48.880550 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f0a3e9cb-0fea-46a1-998a-7a103358d042 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:48.882152 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f0a3e9cb-0fea-46a1-998a-7a103358d042 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:48.919202 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-f0a3e9cb-0fea-46a1-998a-7a103358d042 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.058936 Sep 04 01:09:48.920047 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 415/810] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Sep 4 01:09:48 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 04 01:09:48.952051 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-18fcf2d8-08e6-4b74-8656-daacf952bc94 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:48.966021 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-18fcf2d8-08e6-4b74-8656-daacf952bc94 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a" status: 200 len: 1293 microversion: 2.1 time: 0.166543 Sep 04 01:09:48.983038 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 396/811] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:48 2022] GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a => generated 1293 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:09:49.932895 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-5ac1e7c0-5cb4-48ca-b6b9-4d69fe569da5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:49.934993 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-5ac1e7c0-5cb4-48ca-b6b9-4d69fe569da5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:49.941396 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5ac1e7c0-5cb4-48ca-b6b9-4d69fe569da5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:49.941878 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5ac1e7c0-5cb4-48ca-b6b9-4d69fe569da5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:49.942317 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5ac1e7c0-5cb4-48ca-b6b9-4d69fe569da5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:49.943007 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5ac1e7c0-5cb4-48ca-b6b9-4d69fe569da5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:49.944040 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5ac1e7c0-5cb4-48ca-b6b9-4d69fe569da5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:49.944477 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5ac1e7c0-5cb4-48ca-b6b9-4d69fe569da5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:49.973446 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-5ac1e7c0-5cb4-48ca-b6b9-4d69fe569da5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044853 Sep 04 01:09:49.974004 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 416/812] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Sep 4 01:09:49 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 04 01:09:49.990461 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-0ece091f-589a-487c-a5a1-cc35846d1a2d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:49.991069 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-0ece091f-589a-487c-a5a1-cc35846d1a2d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 16b0498f-4cb8-491a-92cf-dac9890c7b4a] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:49.997554 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0ece091f-589a-487c-a5a1-cc35846d1a2d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:49.998027 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0ece091f-589a-487c-a5a1-cc35846d1a2d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:49.998461 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0ece091f-589a-487c-a5a1-cc35846d1a2d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:49.999139 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0ece091f-589a-487c-a5a1-cc35846d1a2d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:49.999582 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0ece091f-589a-487c-a5a1-cc35846d1a2d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:50.000000 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0ece091f-589a-487c-a5a1-cc35846d1a2d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:50.101231 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-0ece091f-589a-487c-a5a1-cc35846d1a2d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:50.109201 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-0ece091f-589a-487c-a5a1-cc35846d1a2d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a" status: 200 len: 1293 microversion: 2.1 time: 0.123174 Sep 04 01:09:50.109743 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 397/813] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:49 2022] GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a => generated 1293 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:50.994156 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-304c86be-b7b0-45bd-aa74-069e097dd1ff tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:50.994156 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-304c86be-b7b0-45bd-aa74-069e097dd1ff tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:50.996277 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-304c86be-b7b0-45bd-aa74-069e097dd1ff tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:50.997151 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-304c86be-b7b0-45bd-aa74-069e097dd1ff tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:50.997653 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-304c86be-b7b0-45bd-aa74-069e097dd1ff tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:50.998403 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-304c86be-b7b0-45bd-aa74-069e097dd1ff tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:50.998918 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-304c86be-b7b0-45bd-aa74-069e097dd1ff tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:50.999462 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-304c86be-b7b0-45bd-aa74-069e097dd1ff tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:51.029339 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-304c86be-b7b0-45bd-aa74-069e097dd1ff tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.047495 Sep 04 01:09:51.029963 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 417/814] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Sep 4 01:09:50 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3/os-volume_attachments => generated 25 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:51.098686 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-63f72df7-3d44-4c88-9210-1be20aa90ab9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:51.103106 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-63f72df7-3d44-4c88-9210-1be20aa90ab9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:51.106553 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-63f72df7-3d44-4c88-9210-1be20aa90ab9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:51.107037 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-63f72df7-3d44-4c88-9210-1be20aa90ab9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:51.107469 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-63f72df7-3d44-4c88-9210-1be20aa90ab9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:51.108166 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-63f72df7-3d44-4c88-9210-1be20aa90ab9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:51.108611 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-63f72df7-3d44-4c88-9210-1be20aa90ab9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:51.109084 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-63f72df7-3d44-4c88-9210-1be20aa90ab9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:51.132406 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-1581f638-55cb-4e25-a1ad-50930dad43f3 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:51.136998 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-63f72df7-3d44-4c88-9210-1be20aa90ab9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Going to try to terminate instance {{(pid=121037) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 04 01:09:51.139221 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-1581f638-55cb-4e25-a1ad-50930dad43f3 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 16b0498f-4cb8-491a-92cf-dac9890c7b4a] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:51.147542 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1581f638-55cb-4e25-a1ad-50930dad43f3 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:51.148165 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1581f638-55cb-4e25-a1ad-50930dad43f3 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:51.148663 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1581f638-55cb-4e25-a1ad-50930dad43f3 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:51.149469 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1581f638-55cb-4e25-a1ad-50930dad43f3 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:51.149991 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1581f638-55cb-4e25-a1ad-50930dad43f3 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:51.150579 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1581f638-55cb-4e25-a1ad-50930dad43f3 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:51.297747 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-1581f638-55cb-4e25-a1ad-50930dad43f3 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:51.307544 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-1581f638-55cb-4e25-a1ad-50930dad43f3 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a" status: 200 len: 1367 microversion: 2.1 time: 0.179045 Sep 04 01:09:51.308400 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 418/815] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:51 2022] GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a => generated 1367 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:09:51.380783 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-63f72df7-3d44-4c88-9210-1be20aa90ab9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "DELETE /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 204 len: 0 microversion: 2.20 time: 0.285746 Sep 04 01:09:51.381753 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 398/816] 10.222.0.42 () {60 vars in 1280 bytes} [Sun Sep 4 01:09:51 2022] DELETE /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 04 01:09:51.390177 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-e6204db8-0368-4afd-af9e-6d82b059c327 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:51.390944 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-e6204db8-0368-4afd-af9e-6d82b059c327 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:51.398666 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e6204db8-0368-4afd-af9e-6d82b059c327 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:51.399091 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e6204db8-0368-4afd-af9e-6d82b059c327 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:51.399528 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e6204db8-0368-4afd-af9e-6d82b059c327 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:51.400236 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e6204db8-0368-4afd-af9e-6d82b059c327 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:51.400692 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e6204db8-0368-4afd-af9e-6d82b059c327 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:51.401265 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e6204db8-0368-4afd-af9e-6d82b059c327 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:51.562693 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e6204db8-0368-4afd-af9e-6d82b059c327 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:51.572188 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e6204db8-0368-4afd-af9e-6d82b059c327 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:51.573682 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e6204db8-0368-4afd-af9e-6d82b059c327 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:51.576854 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-e6204db8-0368-4afd-af9e-6d82b059c327 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1721 microversion: 2.20 time: 0.188921 Sep 04 01:09:51.576854 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 419/817] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:09:51 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1721 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:09:52.233464 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-3e656808-3ab4-407e-8499-9b4f62134539 req-5c5d5773-afa5-48f7-90f6-3641428d3bdc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "16b0498f-4cb8-491a-92cf-dac9890c7b4a", "tag": "2aad8c1f-5af7-4478-b2c4-782e5042cc6b"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:09:52.244046 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-3e656808-3ab4-407e-8499-9b4f62134539 req-5c5d5773-afa5-48f7-90f6-3641428d3bdc service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:52.244522 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-3e656808-3ab4-407e-8499-9b4f62134539 req-5c5d5773-afa5-48f7-90f6-3641428d3bdc service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:52.244967 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-3e656808-3ab4-407e-8499-9b4f62134539 req-5c5d5773-afa5-48f7-90f6-3641428d3bdc service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:52.259438 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-3e656808-3ab4-407e-8499-9b4f62134539 req-5c5d5773-afa5-48f7-90f6-3641428d3bdc service nova] Creating event network-changed:2aad8c1f-5af7-4478-b2c4-782e5042cc6b for instance 16b0498f-4cb8-491a-92cf-dac9890c7b4a on hv-cinder-maste Sep 04 01:09:52.264501 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-3e656808-3ab4-407e-8499-9b4f62134539 req-5c5d5773-afa5-48f7-90f6-3641428d3bdc service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035117 Sep 04 01:09:52.265175 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 399/818] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:09:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:09:52.331770 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-d117392d-04fa-472d-b96f-3cc5d7e1e7be tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:52.332407 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-d117392d-04fa-472d-b96f-3cc5d7e1e7be tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 16b0498f-4cb8-491a-92cf-dac9890c7b4a] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:52.339685 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d117392d-04fa-472d-b96f-3cc5d7e1e7be tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:52.340160 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d117392d-04fa-472d-b96f-3cc5d7e1e7be tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:52.340593 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d117392d-04fa-472d-b96f-3cc5d7e1e7be tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:52.341313 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d117392d-04fa-472d-b96f-3cc5d7e1e7be tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:52.341768 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d117392d-04fa-472d-b96f-3cc5d7e1e7be tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:52.342193 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d117392d-04fa-472d-b96f-3cc5d7e1e7be tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:52.476295 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-d117392d-04fa-472d-b96f-3cc5d7e1e7be tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:52.483876 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-d117392d-04fa-472d-b96f-3cc5d7e1e7be tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a" status: 200 len: 1367 microversion: 2.1 time: 0.155985 Sep 04 01:09:52.487087 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 420/819] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:52 2022] GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a => generated 1367 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:52.602334 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-4986ab63-48f6-44e2-8b47-9dab63112016 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:52.603022 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-4986ab63-48f6-44e2-8b47-9dab63112016 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:52.609661 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4986ab63-48f6-44e2-8b47-9dab63112016 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:52.610135 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4986ab63-48f6-44e2-8b47-9dab63112016 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:52.610565 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4986ab63-48f6-44e2-8b47-9dab63112016 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:52.611248 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4986ab63-48f6-44e2-8b47-9dab63112016 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:52.611936 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4986ab63-48f6-44e2-8b47-9dab63112016 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:52.612363 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4986ab63-48f6-44e2-8b47-9dab63112016 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:52.745952 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-4986ab63-48f6-44e2-8b47-9dab63112016 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:52.754206 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-4986ab63-48f6-44e2-8b47-9dab63112016 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:52.755764 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-4986ab63-48f6-44e2-8b47-9dab63112016 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:52.759068 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-4986ab63-48f6-44e2-8b47-9dab63112016 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1721 microversion: 2.20 time: 0.159674 Sep 04 01:09:52.759068 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 400/820] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:09:52 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1721 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:09:53.507944 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b153834b-d47d-4340-a480-b6cd85e88747 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:53.508566 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-b153834b-d47d-4340-a480-b6cd85e88747 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 16b0498f-4cb8-491a-92cf-dac9890c7b4a] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:53.515063 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b153834b-d47d-4340-a480-b6cd85e88747 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:53.515884 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b153834b-d47d-4340-a480-b6cd85e88747 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:53.516494 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b153834b-d47d-4340-a480-b6cd85e88747 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:53.517252 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b153834b-d47d-4340-a480-b6cd85e88747 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:53.517710 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b153834b-d47d-4340-a480-b6cd85e88747 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:53.518133 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b153834b-d47d-4340-a480-b6cd85e88747 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:53.672356 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-b153834b-d47d-4340-a480-b6cd85e88747 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:53.682082 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-b153834b-d47d-4340-a480-b6cd85e88747 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a" status: 200 len: 1367 microversion: 2.1 time: 0.178182 Sep 04 01:09:53.682675 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 421/821] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:53 2022] GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a => generated 1367 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:53.782952 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-73f3ae85-ea07-4d6e-8495-d015aa177926 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:53.783589 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-73f3ae85-ea07-4d6e-8495-d015aa177926 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:53.791239 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-73f3ae85-ea07-4d6e-8495-d015aa177926 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:53.791716 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-73f3ae85-ea07-4d6e-8495-d015aa177926 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:53.792222 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-73f3ae85-ea07-4d6e-8495-d015aa177926 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:53.792951 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-73f3ae85-ea07-4d6e-8495-d015aa177926 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:53.793422 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-73f3ae85-ea07-4d6e-8495-d015aa177926 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:53.793924 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-73f3ae85-ea07-4d6e-8495-d015aa177926 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:54.154730 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-73f3ae85-ea07-4d6e-8495-d015aa177926 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:54.163084 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-73f3ae85-ea07-4d6e-8495-d015aa177926 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:54.164603 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-73f3ae85-ea07-4d6e-8495-d015aa177926 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:54.165796 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-73f3ae85-ea07-4d6e-8495-d015aa177926 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1721 microversion: 2.20 time: 0.386848 Sep 04 01:09:54.166356 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 401/822] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:09:53 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1721 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:09:54.705766 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-d565fc05-0aed-4ef1-9978-f26c986d0669 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:54.706680 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-d565fc05-0aed-4ef1-9978-f26c986d0669 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 16b0498f-4cb8-491a-92cf-dac9890c7b4a] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:54.714689 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d565fc05-0aed-4ef1-9978-f26c986d0669 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:54.715516 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d565fc05-0aed-4ef1-9978-f26c986d0669 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:54.716180 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d565fc05-0aed-4ef1-9978-f26c986d0669 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:54.717407 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d565fc05-0aed-4ef1-9978-f26c986d0669 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:54.718078 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d565fc05-0aed-4ef1-9978-f26c986d0669 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:54.718731 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d565fc05-0aed-4ef1-9978-f26c986d0669 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:54.723701 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:09:54.760013 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-f977fa84-8dfa-4ecd-99bd-8180b2027aa9 req-2a43c126-e0a5-43f1-bfd9-b0bd8aab799f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "49be5a0b-f748-486e-a546-c07a8e8451a3", "name": "network-vif-deleted", "tag": "fcbc60fd-1785-4f94-9436-a520ec06cdaf"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:09:54.769243 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-f977fa84-8dfa-4ecd-99bd-8180b2027aa9 req-2a43c126-e0a5-43f1-bfd9-b0bd8aab799f service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:54.769884 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-f977fa84-8dfa-4ecd-99bd-8180b2027aa9 req-2a43c126-e0a5-43f1-bfd9-b0bd8aab799f service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:54.770444 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-f977fa84-8dfa-4ecd-99bd-8180b2027aa9 req-2a43c126-e0a5-43f1-bfd9-b0bd8aab799f service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:54.783838 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-f977fa84-8dfa-4ecd-99bd-8180b2027aa9 req-2a43c126-e0a5-43f1-bfd9-b0bd8aab799f service nova] Creating event network-vif-deleted:fcbc60fd-1785-4f94-9436-a520ec06cdaf for instance 49be5a0b-f748-486e-a546-c07a8e8451a3 on hv-cinder-maste Sep 04 01:09:54.789901 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-f977fa84-8dfa-4ecd-99bd-8180b2027aa9 req-2a43c126-e0a5-43f1-bfd9-b0bd8aab799f service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040072 Sep 04 01:09:54.791118 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 402/823] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:09:54 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:09:54.869772 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-d565fc05-0aed-4ef1-9978-f26c986d0669 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:54.879003 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-d565fc05-0aed-4ef1-9978-f26c986d0669 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a" status: 200 len: 1367 microversion: 2.1 time: 0.177102 Sep 04 01:09:54.879592 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 422/824] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:54 2022] GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a => generated 1367 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:55.192626 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-4aed3225-b82c-4d44-9f59-e58c3cec0722 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:55.193465 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-4aed3225-b82c-4d44-9f59-e58c3cec0722 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:55.200547 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4aed3225-b82c-4d44-9f59-e58c3cec0722 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:55.201016 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4aed3225-b82c-4d44-9f59-e58c3cec0722 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:55.201482 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4aed3225-b82c-4d44-9f59-e58c3cec0722 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:55.202175 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4aed3225-b82c-4d44-9f59-e58c3cec0722 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:55.202627 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4aed3225-b82c-4d44-9f59-e58c3cec0722 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:55.203057 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4aed3225-b82c-4d44-9f59-e58c3cec0722 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:55.207142 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:09:55.266550 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-4aed3225-b82c-4d44-9f59-e58c3cec0722 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:55.277811 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-4aed3225-b82c-4d44-9f59-e58c3cec0722 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:55.277811 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-4aed3225-b82c-4d44-9f59-e58c3cec0722 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:55.278177 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-4aed3225-b82c-4d44-9f59-e58c3cec0722 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1654 microversion: 2.20 time: 0.089828 Sep 04 01:09:55.278809 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 403/825] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:09:55 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1654 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:09:55.905815 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-d7e252eb-e98c-4e7b-ab79-980e804783b0 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:55.906431 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-d7e252eb-e98c-4e7b-ab79-980e804783b0 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 16b0498f-4cb8-491a-92cf-dac9890c7b4a] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:55.916104 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d7e252eb-e98c-4e7b-ab79-980e804783b0 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:55.916594 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d7e252eb-e98c-4e7b-ab79-980e804783b0 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:55.917844 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d7e252eb-e98c-4e7b-ab79-980e804783b0 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:55.918555 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d7e252eb-e98c-4e7b-ab79-980e804783b0 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:55.919009 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d7e252eb-e98c-4e7b-ab79-980e804783b0 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:55.919738 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d7e252eb-e98c-4e7b-ab79-980e804783b0 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:56.171130 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-d7e252eb-e98c-4e7b-ab79-980e804783b0 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:56.179107 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-d7e252eb-e98c-4e7b-ab79-980e804783b0 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a" status: 200 len: 1367 microversion: 2.1 time: 0.279763 Sep 04 01:09:56.179681 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 423/826] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:55 2022] GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a => generated 1367 bytes in 281 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:56.304660 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-f22e4bf0-e4fc-4b34-8b51-efd8539b8d22 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:56.305295 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-f22e4bf0-e4fc-4b34-8b51-efd8539b8d22 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:56.312164 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f22e4bf0-e4fc-4b34-8b51-efd8539b8d22 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:56.312636 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f22e4bf0-e4fc-4b34-8b51-efd8539b8d22 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:56.313084 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f22e4bf0-e4fc-4b34-8b51-efd8539b8d22 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:56.313774 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f22e4bf0-e4fc-4b34-8b51-efd8539b8d22 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:56.314221 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f22e4bf0-e4fc-4b34-8b51-efd8539b8d22 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:56.314638 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f22e4bf0-e4fc-4b34-8b51-efd8539b8d22 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:56.347165 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f22e4bf0-e4fc-4b34-8b51-efd8539b8d22 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:56.347645 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f22e4bf0-e4fc-4b34-8b51-efd8539b8d22 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:56.348079 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f22e4bf0-e4fc-4b34-8b51-efd8539b8d22 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:56.348566 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.objects.instance [None req-f22e4bf0-e4fc-4b34-8b51-efd8539b8d22 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lazy-loading 'fault' on Instance uuid 49be5a0b-f748-486e-a546-c07a8e8451a3 {{(pid=121037) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:09:56.444744 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-f22e4bf0-e4fc-4b34-8b51-efd8539b8d22 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:56.452867 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-f22e4bf0-e4fc-4b34-8b51-efd8539b8d22 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:56.454372 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-f22e4bf0-e4fc-4b34-8b51-efd8539b8d22 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:56.455523 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-f22e4bf0-e4fc-4b34-8b51-efd8539b8d22 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 200 len: 1371 microversion: 2.20 time: 0.154919 Sep 04 01:09:56.456083 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 404/827] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:09:56 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 1371 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:09:57.202407 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-6b5ecc39-23ce-434d-88ad-bd92f39d9eeb tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:57.203040 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-6b5ecc39-23ce-434d-88ad-bd92f39d9eeb tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 16b0498f-4cb8-491a-92cf-dac9890c7b4a] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:57.204850 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell0:SELECT=2 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:09:57.205513 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell0:SELECT=10 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:09:57.211397 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6b5ecc39-23ce-434d-88ad-bd92f39d9eeb tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:57.211912 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6b5ecc39-23ce-434d-88ad-bd92f39d9eeb tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:57.212344 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6b5ecc39-23ce-434d-88ad-bd92f39d9eeb tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:57.213021 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6b5ecc39-23ce-434d-88ad-bd92f39d9eeb tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:57.213564 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6b5ecc39-23ce-434d-88ad-bd92f39d9eeb tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:57.213981 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6b5ecc39-23ce-434d-88ad-bd92f39d9eeb tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:57.341711 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-6b5ecc39-23ce-434d-88ad-bd92f39d9eeb tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:57.349340 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-6b5ecc39-23ce-434d-88ad-bd92f39d9eeb tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a" status: 200 len: 1367 microversion: 2.1 time: 0.150758 Sep 04 01:09:57.349904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 424/828] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:57 2022] GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a => generated 1367 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:57.485918 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-a289d14d-dd34-4db3-8502-d06e4798a64e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:57.486567 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-a289d14d-dd34-4db3-8502-d06e4798a64e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:57.488648 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell0:SELECT=2 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:09:57.494001 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a289d14d-dd34-4db3-8502-d06e4798a64e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:57.494473 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a289d14d-dd34-4db3-8502-d06e4798a64e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:57.495122 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a289d14d-dd34-4db3-8502-d06e4798a64e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:57.495989 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a289d14d-dd34-4db3-8502-d06e4798a64e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:57.496430 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a289d14d-dd34-4db3-8502-d06e4798a64e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:57.496847 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a289d14d-dd34-4db3-8502-d06e4798a64e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:57.511399 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.wsgi [None req-a289d14d-dd34-4db3-8502-d06e4798a64e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] HTTP exception thrown: Instance 49be5a0b-f748-486e-a546-c07a8e8451a3 could not be found. Sep 04 01:09:57.513048 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-a289d14d-dd34-4db3-8502-d06e4798a64e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Request method failure captured: Sep 04 01:09:57.513048 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: request: GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 HTTP/1.1 Sep 04 01:09:57.513048 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept: application/json Sep 04 01:09:57.513048 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept-Encoding: identity Sep 04 01:09:57.513048 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Connection: close Sep 04 01:09:57.513048 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Length: 0 Sep 04 01:09:57.513048 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Type: application/json Sep 04 01:09:57.513048 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Host: 10.222.0.42 Sep 04 01:09:57.513048 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Openstack-System-Scope: None Sep 04 01:09:57.513048 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: User-Agent: python-urllib3/1.26.12 Sep 04 01:09:57.513048 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Auth-Token: gAAAAABjE_nKpA1VnxteF8h4QX2ISAeVrHzayiTS_h5gb61lOBjA6-Oug7W0SDbkxntdMAdvczgEZhwiDWZ1krkg1A5Y324MQzG5qSTO_8yWZlwGNhCpf9OdVOm9oT4bo33eHTJzvu7g_Ei9xW9o9uHJDzE5TNj7NIiad4pBdXs2jBxLyTcMcQ0 Sep 04 01:09:57.513048 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Id: None Sep 04 01:09:57.513048 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Name: None Sep 04 01:09:57.513048 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Identity-Status: Confirmed Sep 04 01:09:57.513048 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Is-Admin-Project: True Sep 04 01:09:57.513048 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Openstack-Nova-Api-Version: 2.20 Sep 04 01:09:57.513048 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Id: default Sep 04 01:09:57.513048 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Name: Default Sep 04 01:09:57.513048 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Id: 55faed42b7f243df9df879d581797f56 Sep 04 01:09:57.513048 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1588860844 Sep 04 01:09:57.513048 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Role: reader,member Sep 04 01:09:57.513048 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Roles: reader,member Sep 04 01:09:57.514519 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/55faed42b7f243df9df879d581797f56"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_55faed42b7f243df9df879d581797f56", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/55faed42b7f243df9df879d581797f56"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/55faed42b7f243df9df879d581797f56"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:09:57.514519 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1588860844 Sep 04 01:09:57.514519 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Id: 55faed42b7f243df9df879d581797f56 Sep 04 01:09:57.514519 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1588860844 Sep 04 01:09:57.514519 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User: tempest-AttachVolumeShelveTestJSON-1588860844-project Sep 04 01:09:57.514519 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Id: default Sep 04 01:09:57.514519 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Name: Default Sep 04 01:09:57.514519 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Id: fbcd4c19e766446188e3b3f41a44e2e4 Sep 04 01:09:57.514519 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1588860844-project Sep 04 01:09:57.514519 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: method: > Sep 04 01:09:57.514519 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: exception: Instance 49be5a0b-f748-486e-a546-c07a8e8451a3 could not be found. Sep 04 01:09:57.514519 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: action_args: {'id': '49be5a0b-f748-486e-a546-c07a8e8451a3'} Sep 04 01:09:57.515256 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:09:57.515256 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:09:57.515256 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:09:57.515256 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:09:57.515256 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:09:57.515256 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:09:57.515256 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:09:57.515256 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:09:57.515256 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:09:57.515256 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:09:57.515256 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 49be5a0b-f748-486e-a546-c07a8e8451a3 could not be found. Sep 04 01:09:57.515256 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:09:57.515256 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:09:57.515256 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:09:57.515256 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:09:57.515256 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:09:57.515256 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:09:57.515256 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:09:57.515256 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:09:57.515256 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:09:57.516711 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:09:57.516711 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 04 01:09:57.516711 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 04 01:09:57.516711 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:09:57.516711 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:09:57.516711 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:09:57.516711 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:09:57.516711 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 49be5a0b-f748-486e-a546-c07a8e8451a3 could not be found. Sep 04 01:09:57.516711 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:09:57.516711 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:09:57.516711 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:09:57.516711 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:09:57.516711 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:09:57.516711 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:09:57.516711 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:09:57.516711 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:09:57.516711 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 49be5a0b-f748-486e-a546-c07a8e8451a3 could not be found. Sep 04 01:09:57.516711 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:09:57.518054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-a289d14d-dd34-4db3-8502-d06e4798a64e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Returning 404 to user: Instance 49be5a0b-f748-486e-a546-c07a8e8451a3 could not be found. {{(pid=121037) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:09:57.518054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-a289d14d-dd34-4db3-8502-d06e4798a64e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 404 len: 111 microversion: 2.20 time: 0.033023 Sep 04 01:09:57.518054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 405/829] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:09:57 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 04 01:09:57.524426 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-f2bf9f4e-0859-48a6-a6e7-3e78ee10535c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '.version_select at 0x7fc10c4f9940>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:57.651159 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-f2bf9f4e-0859-48a6-a6e7-3e78ee10535c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-870969389" status: 204 len: 0 microversion: 2.20 time: 0.131553 Sep 04 01:09:57.651784 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 425/830] 10.222.0.42 () {60 vars in 1259 bytes} [Sun Sep 4 01:09:57 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-870969389 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 04 01:09:58.375660 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-d6d62cab-fde9-4ec5-86e3-f6b02b87ab2e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:58.376275 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-d6d62cab-fde9-4ec5-86e3-f6b02b87ab2e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 16b0498f-4cb8-491a-92cf-dac9890c7b4a] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:58.383907 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d6d62cab-fde9-4ec5-86e3-f6b02b87ab2e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:58.384396 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d6d62cab-fde9-4ec5-86e3-f6b02b87ab2e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:58.384810 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d6d62cab-fde9-4ec5-86e3-f6b02b87ab2e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:58.385552 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d6d62cab-fde9-4ec5-86e3-f6b02b87ab2e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:58.386005 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d6d62cab-fde9-4ec5-86e3-f6b02b87ab2e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:58.386432 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d6d62cab-fde9-4ec5-86e3-f6b02b87ab2e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:58.414273 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-ebaa8ba6-1c1d-4f25-a08a-43f4152992d4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Action: 'create', calling method: .version_select at 0x7fc10c4e4280>, body: {"keypair": {"name": "tempest-keypair-419803257", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCXyjNUo0bfnmKROgwwX2MZFBj1BdREWij33elEsNGRO9djtTtGRDYeTrCnxvIAAF5rniC4yJgDDqQMPhUkkbg3iUcIVuFlv6chZupKlJSJIBANGHiHr7BWExAllNSPrSQ=="}} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:09:58.439499 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-ebaa8ba6-1c1d-4f25-a08a-43f4152992d4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Getting quotas for project 55faed42b7f243df9df879d581797f56. Resources: {'key_pairs'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 04 01:09:58.443954 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-ebaa8ba6-1c1d-4f25-a08a-43f4152992d4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Getting quotas for user fbcd4c19e766446188e3b3f41a44e2e4 and project 55faed42b7f243df9df879d581797f56. Resources: {'key_pairs'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 04 01:09:58.523233 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-d6d62cab-fde9-4ec5-86e3-f6b02b87ab2e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:58.530958 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-d6d62cab-fde9-4ec5-86e3-f6b02b87ab2e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a" status: 200 len: 1367 microversion: 2.1 time: 0.159817 Sep 04 01:09:58.531525 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 406/831] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:58 2022] GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a => generated 1367 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:09:58.547049 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-ebaa8ba6-1c1d-4f25-a08a-43f4152992d4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.136496 Sep 04 01:09:58.547614 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 426/832] 10.222.0.42 () {62 vars in 1200 bytes} [Sun Sep 4 01:09:58 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 138 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Sep 04 01:09:59.555690 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-653c45a5-b3a6-4515-8afe-45ecbf070b17 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:09:59.556294 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-653c45a5-b3a6-4515-8afe-45ecbf070b17 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 16b0498f-4cb8-491a-92cf-dac9890c7b4a] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:09:59.563122 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-653c45a5-b3a6-4515-8afe-45ecbf070b17 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:59.563617 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-653c45a5-b3a6-4515-8afe-45ecbf070b17 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:59.564058 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-653c45a5-b3a6-4515-8afe-45ecbf070b17 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:59.564760 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-653c45a5-b3a6-4515-8afe-45ecbf070b17 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:09:59.565256 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-653c45a5-b3a6-4515-8afe-45ecbf070b17 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:09:59.565683 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-653c45a5-b3a6-4515-8afe-45ecbf070b17 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:09:59.769857 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-653c45a5-b3a6-4515-8afe-45ecbf070b17 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:09:59.777888 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-653c45a5-b3a6-4515-8afe-45ecbf070b17 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a" status: 200 len: 1367 microversion: 2.1 time: 0.227034 Sep 04 01:09:59.778976 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 407/833] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:09:59 2022] GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a => generated 1367 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:00.807309 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-98109afa-0d54-461b-bd48-95bfe66837d2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:00.808471 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-98109afa-0d54-461b-bd48-95bfe66837d2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 16b0498f-4cb8-491a-92cf-dac9890c7b4a] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:00.816223 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-4ef555bb-7969-464e-99bd-671fc119d902 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-733183972", "imageRef": "a11315c3-ce81-478b-91e4-5353f189dfa7", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "3e702e6e-ebe1-46a7-b30d-f3d140ac2993"}], "security_groups": [{"name": "tempest-securitygroup--946139031"}], "key_name": "tempest-keypair-419803257", "user_data": "}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:10:00.830571 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-98109afa-0d54-461b-bd48-95bfe66837d2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:00.831058 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-98109afa-0d54-461b-bd48-95bfe66837d2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:00.831485 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-98109afa-0d54-461b-bd48-95bfe66837d2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:00.832395 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-98109afa-0d54-461b-bd48-95bfe66837d2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:00.833019 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-98109afa-0d54-461b-bd48-95bfe66837d2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:00.833521 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-98109afa-0d54-461b-bd48-95bfe66837d2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:00.955767 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.network.neutron [None req-4ef555bb-7969-464e-99bd-671fc119d902 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] validate_networks() for [('3e702e6e-ebe1-46a7-b30d-f3d140ac2993', None, None, None, None, None)] {{(pid=121037) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 04 01:10:01.020798 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-98109afa-0d54-461b-bd48-95bfe66837d2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:01.030957 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-98109afa-0d54-461b-bd48-95bfe66837d2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a" status: 200 len: 1367 microversion: 2.1 time: 0.233074 Sep 04 01:10:01.031554 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 427/834] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:00 2022] GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a => generated 1367 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:01.057004 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-6f58e8c3-6297-4906-a9e0-409a34daa674 req-0f907ca9-d592-4dc1-a2e8-4bb47b02f928 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "16b0498f-4cb8-491a-92cf-dac9890c7b4a", "name": "network-vif-plugged", "status": "completed", "tag": "2aad8c1f-5af7-4478-b2c4-782e5042cc6b"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:10:01.067077 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-6f58e8c3-6297-4906-a9e0-409a34daa674 req-0f907ca9-d592-4dc1-a2e8-4bb47b02f928 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:01.067554 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-6f58e8c3-6297-4906-a9e0-409a34daa674 req-0f907ca9-d592-4dc1-a2e8-4bb47b02f928 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:01.068051 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-6f58e8c3-6297-4906-a9e0-409a34daa674 req-0f907ca9-d592-4dc1-a2e8-4bb47b02f928 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:01.079015 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-6f58e8c3-6297-4906-a9e0-409a34daa674 req-0f907ca9-d592-4dc1-a2e8-4bb47b02f928 service nova] Creating event network-vif-plugged:2aad8c1f-5af7-4478-b2c4-782e5042cc6b for instance 16b0498f-4cb8-491a-92cf-dac9890c7b4a on hv-cinder-maste Sep 04 01:10:01.084032 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-6f58e8c3-6297-4906-a9e0-409a34daa674 req-0f907ca9-d592-4dc1-a2e8-4bb47b02f928 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032337 Sep 04 01:10:01.085306 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 428/835] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:10:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:10:01.693670 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-4ef555bb-7969-464e-99bd-671fc119d902 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Flavor limits 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 04 01:10:01.694762 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-4ef555bb-7969-464e-99bd-671fc119d902 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Image limits 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 04 01:10:01.695319 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-4ef555bb-7969-464e-99bd-671fc119d902 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Flavor pref 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 04 01:10:01.695734 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-4ef555bb-7969-464e-99bd-671fc119d902 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Image pref 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 04 01:10:01.696152 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-4ef555bb-7969-464e-99bd-671fc119d902 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 04 01:10:01.705048 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4ef555bb-7969-464e-99bd-671fc119d902 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:01.705552 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4ef555bb-7969-464e-99bd-671fc119d902 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:01.705979 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4ef555bb-7969-464e-99bd-671fc119d902 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:01.706853 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4ef555bb-7969-464e-99bd-671fc119d902 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:01.707297 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4ef555bb-7969-464e-99bd-671fc119d902 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:01.707716 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4ef555bb-7969-464e-99bd-671fc119d902 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:01.724005 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.quota [None req-4ef555bb-7969-464e-99bd-671fc119d902 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Getting quotas for project 55faed42b7f243df9df879d581797f56. Resources: {'ram', 'instances', 'cores'} {{(pid=121037) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 04 01:10:01.727915 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.quota [None req-4ef555bb-7969-464e-99bd-671fc119d902 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Getting quotas for user fbcd4c19e766446188e3b3f41a44e2e4 and project 55faed42b7f243df9df879d581797f56. Resources: {'ram', 'instances', 'cores'} {{(pid=121037) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 04 01:10:01.739031 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-4ef555bb-7969-464e-99bd-671fc119d902 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Going to run 1 instances... {{(pid=121037) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 04 01:10:01.742010 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-4ef555bb-7969-464e-99bd-671fc119d902 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a11315c3-ce81-478b-91e4-5353f189dfa7',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121037) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 04 01:10:01.856779 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-4ef555bb-7969-464e-99bd-671fc119d902 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.20 time: 1.044654 Sep 04 01:10:01.858290 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 408/836] 10.222.0.42 () {62 vars in 1188 bytes} [Sun Sep 4 01:10:00 2022] POST /compute/v2.1/servers => generated 401 bytes in 1046 msecs (HTTP/1.1 202) 10 headers in 453 bytes (2 switches on core 0) Sep 04 01:10:01.871560 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-fc66d35a-5f0b-45d4-a85d-5944ffd4f561 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:01.872253 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-fc66d35a-5f0b-45d4-a85d-5944ffd4f561 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:01.926677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-fc66d35a-5f0b-45d4-a85d-5944ffd4f561 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:01.934900 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-fc66d35a-5f0b-45d4-a85d-5944ffd4f561 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:01.936337 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-fc66d35a-5f0b-45d4-a85d-5944ffd4f561 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:01.937461 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-fc66d35a-5f0b-45d4-a85d-5944ffd4f561 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1280 microversion: 2.20 time: 0.071980 Sep 04 01:10:01.938014 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 429/837] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:10:01 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1280 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:10:02.058294 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-4a792768-9637-4072-81c0-844767c14a17 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:02.058909 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-4a792768-9637-4072-81c0-844767c14a17 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 16b0498f-4cb8-491a-92cf-dac9890c7b4a] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:02.065063 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4a792768-9637-4072-81c0-844767c14a17 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:02.065572 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4a792768-9637-4072-81c0-844767c14a17 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:02.066007 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4a792768-9637-4072-81c0-844767c14a17 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:02.066699 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4a792768-9637-4072-81c0-844767c14a17 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:02.067146 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4a792768-9637-4072-81c0-844767c14a17 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:02.067572 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4a792768-9637-4072-81c0-844767c14a17 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:02.195760 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-4a792768-9637-4072-81c0-844767c14a17 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:02.203450 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-4a792768-9637-4072-81c0-844767c14a17 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a" status: 200 len: 1367 microversion: 2.1 time: 0.149960 Sep 04 01:10:02.203990 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 409/838] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:02 2022] GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a => generated 1367 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:02.963587 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-e2ae536c-dd02-426f-bc6c-22c78a8a4851 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:02.964496 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-e2ae536c-dd02-426f-bc6c-22c78a8a4851 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:02.972376 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e2ae536c-dd02-426f-bc6c-22c78a8a4851 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:02.973041 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e2ae536c-dd02-426f-bc6c-22c78a8a4851 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:02.973593 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e2ae536c-dd02-426f-bc6c-22c78a8a4851 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:02.974286 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e2ae536c-dd02-426f-bc6c-22c78a8a4851 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:02.974745 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e2ae536c-dd02-426f-bc6c-22c78a8a4851 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:02.975178 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e2ae536c-dd02-426f-bc6c-22c78a8a4851 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:03.038777 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e2ae536c-dd02-426f-bc6c-22c78a8a4851 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:03.046949 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e2ae536c-dd02-426f-bc6c-22c78a8a4851 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:03.048407 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e2ae536c-dd02-426f-bc6c-22c78a8a4851 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:03.049540 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-e2ae536c-dd02-426f-bc6c-22c78a8a4851 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1284 microversion: 2.20 time: 0.090356 Sep 04 01:10:03.050097 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 430/839] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:10:02 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1284 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:10:03.226405 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-c64ad657-91e3-4d69-9176-77d372025600 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:03.227011 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-c64ad657-91e3-4d69-9176-77d372025600 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 16b0498f-4cb8-491a-92cf-dac9890c7b4a] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:03.233455 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c64ad657-91e3-4d69-9176-77d372025600 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:03.233934 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c64ad657-91e3-4d69-9176-77d372025600 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:03.234367 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c64ad657-91e3-4d69-9176-77d372025600 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:03.235057 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c64ad657-91e3-4d69-9176-77d372025600 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:03.235508 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c64ad657-91e3-4d69-9176-77d372025600 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:03.235930 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c64ad657-91e3-4d69-9176-77d372025600 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:03.364818 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-c64ad657-91e3-4d69-9176-77d372025600 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:03.373294 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-c64ad657-91e3-4d69-9176-77d372025600 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a" status: 200 len: 1557 microversion: 2.1 time: 0.150757 Sep 04 01:10:03.373861 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 410/840] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:03 2022] GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a => generated 1557 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:03.396086 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-e9247b80-3673-4ede-8b7e-8e14d26c1716 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:03.396579 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-e9247b80-3673-4ede-8b7e-8e14d26c1716 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 16b0498f-4cb8-491a-92cf-dac9890c7b4a] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:03.402484 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e9247b80-3673-4ede-8b7e-8e14d26c1716 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:03.402963 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e9247b80-3673-4ede-8b7e-8e14d26c1716 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:03.403400 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e9247b80-3673-4ede-8b7e-8e14d26c1716 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:03.404086 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e9247b80-3673-4ede-8b7e-8e14d26c1716 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:03.404534 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e9247b80-3673-4ede-8b7e-8e14d26c1716 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:03.404959 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e9247b80-3673-4ede-8b7e-8e14d26c1716 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:03.484518 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-e9247b80-3673-4ede-8b7e-8e14d26c1716 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.092179 Sep 04 01:10:03.485144 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 431/841] 10.222.0.42 () {58 vars in 1275 bytes} [Sun Sep 4 01:10:03 2022] GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a/os-interface => generated 284 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:10:04.075293 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-9742eeb6-294f-4641-9016-8bcdab1146f7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:04.075907 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-9742eeb6-294f-4641-9016-8bcdab1146f7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:04.082251 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9742eeb6-294f-4641-9016-8bcdab1146f7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:04.086438 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9742eeb6-294f-4641-9016-8bcdab1146f7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:04.086438 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9742eeb6-294f-4641-9016-8bcdab1146f7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:04.086438 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9742eeb6-294f-4641-9016-8bcdab1146f7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:04.086438 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9742eeb6-294f-4641-9016-8bcdab1146f7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:04.086815 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9742eeb6-294f-4641-9016-8bcdab1146f7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:04.160701 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-9742eeb6-294f-4641-9016-8bcdab1146f7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:04.172247 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-9742eeb6-294f-4641-9016-8bcdab1146f7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:04.173803 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-9742eeb6-294f-4641-9016-8bcdab1146f7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:04.178848 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-9742eeb6-294f-4641-9016-8bcdab1146f7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1332 microversion: 2.20 time: 0.103798 Sep 04 01:10:04.178848 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 411/842] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:10:04 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1332 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:10:04.561557 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-15f7a242-6e29-4d57-8d95-067fd682f876 req-90186408-f174-4a87-943b-03d86a94605c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "16b0498f-4cb8-491a-92cf-dac9890c7b4a", "tag": "2aad8c1f-5af7-4478-b2c4-782e5042cc6b"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:10:04.571912 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-15f7a242-6e29-4d57-8d95-067fd682f876 req-90186408-f174-4a87-943b-03d86a94605c service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:04.572410 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-15f7a242-6e29-4d57-8d95-067fd682f876 req-90186408-f174-4a87-943b-03d86a94605c service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:04.573314 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-15f7a242-6e29-4d57-8d95-067fd682f876 req-90186408-f174-4a87-943b-03d86a94605c service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:04.583899 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-15f7a242-6e29-4d57-8d95-067fd682f876 req-90186408-f174-4a87-943b-03d86a94605c service nova] Creating event network-changed:2aad8c1f-5af7-4478-b2c4-782e5042cc6b for instance 16b0498f-4cb8-491a-92cf-dac9890c7b4a on hv-cinder-maste Sep 04 01:10:04.589572 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-15f7a242-6e29-4d57-8d95-067fd682f876 req-90186408-f174-4a87-943b-03d86a94605c service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032065 Sep 04 01:10:04.590182 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 432/843] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:10:04 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:10:04.763254 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-aa5532b8-f322-4d69-979e-669161233973 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:04.765818 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-aa5532b8-f322-4d69-979e-669161233973 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:04.775023 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-aa5532b8-f322-4d69-979e-669161233973 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:04.775788 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-aa5532b8-f322-4d69-979e-669161233973 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:04.785880 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-aa5532b8-f322-4d69-979e-669161233973 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:04.785880 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-aa5532b8-f322-4d69-979e-669161233973 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:04.785880 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-aa5532b8-f322-4d69-979e-669161233973 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:04.785880 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-aa5532b8-f322-4d69-979e-669161233973 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:04.814234 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-aa5532b8-f322-4d69-979e-669161233973 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.055090 Sep 04 01:10:04.821032 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 412/844] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 4 01:10:04 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2/os-volume_attachments => generated 25 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 04 01:10:04.891407 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b6b05407-3d18-4705-9bdd-e356da08b2e1 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:04.892370 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-b6b05407-3d18-4705-9bdd-e356da08b2e1 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:04.899739 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b6b05407-3d18-4705-9bdd-e356da08b2e1 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:04.900266 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b6b05407-3d18-4705-9bdd-e356da08b2e1 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:04.900641 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b6b05407-3d18-4705-9bdd-e356da08b2e1 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:04.901374 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b6b05407-3d18-4705-9bdd-e356da08b2e1 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:04.901831 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b6b05407-3d18-4705-9bdd-e356da08b2e1 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:04.902259 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b6b05407-3d18-4705-9bdd-e356da08b2e1 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:04.934406 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-b6b05407-3d18-4705-9bdd-e356da08b2e1 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Going to try to terminate instance {{(pid=121036) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 04 01:10:05.158995 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-b6b05407-3d18-4705-9bdd-e356da08b2e1 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "DELETE /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 204 len: 0 microversion: 2.1 time: 0.271447 Sep 04 01:10:05.160427 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 433/845] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 4 01:10:04 2022] DELETE /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 04 01:10:05.169576 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-3b6c6497-f62d-45d1-b962-3f25128ae84e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:05.170672 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-3b6c6497-f62d-45d1-b962-3f25128ae84e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:05.177041 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3b6c6497-f62d-45d1-b962-3f25128ae84e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:05.181272 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3b6c6497-f62d-45d1-b962-3f25128ae84e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:05.182187 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3b6c6497-f62d-45d1-b962-3f25128ae84e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:05.184031 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3b6c6497-f62d-45d1-b962-3f25128ae84e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:05.184888 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3b6c6497-f62d-45d1-b962-3f25128ae84e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:05.185787 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3b6c6497-f62d-45d1-b962-3f25128ae84e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:05.201721 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-6352649a-bd2c-4efb-a30f-9a2f7a17deee tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:05.202525 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-6352649a-bd2c-4efb-a30f-9a2f7a17deee tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:05.211467 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6352649a-bd2c-4efb-a30f-9a2f7a17deee tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:05.212110 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6352649a-bd2c-4efb-a30f-9a2f7a17deee tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:05.212647 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6352649a-bd2c-4efb-a30f-9a2f7a17deee tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:05.213689 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6352649a-bd2c-4efb-a30f-9a2f7a17deee tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:05.214352 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6352649a-bd2c-4efb-a30f-9a2f7a17deee tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:05.215129 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6352649a-bd2c-4efb-a30f-9a2f7a17deee tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:05.313135 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-6352649a-bd2c-4efb-a30f-9a2f7a17deee tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:05.322723 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-6352649a-bd2c-4efb-a30f-9a2f7a17deee tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:05.324164 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-6352649a-bd2c-4efb-a30f-9a2f7a17deee tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:05.325315 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-6352649a-bd2c-4efb-a30f-9a2f7a17deee tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1350 microversion: 2.20 time: 0.128280 Sep 04 01:10:05.326093 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 434/846] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:10:05 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1350 bytes in 130 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:10:05.348034 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-3b6c6497-f62d-45d1-b962-3f25128ae84e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:05.356097 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-3b6c6497-f62d-45d1-b962-3f25128ae84e tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1670 microversion: 2.1 time: 0.190698 Sep 04 01:10:05.356666 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 413/847] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:05 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1670 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:06.350592 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-a5d110a6-ebbc-4f71-889b-b6b562140ee7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:06.351221 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-a5d110a6-ebbc-4f71-889b-b6b562140ee7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:06.357761 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a5d110a6-ebbc-4f71-889b-b6b562140ee7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:06.358233 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a5d110a6-ebbc-4f71-889b-b6b562140ee7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:06.358663 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a5d110a6-ebbc-4f71-889b-b6b562140ee7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:06.359339 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a5d110a6-ebbc-4f71-889b-b6b562140ee7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:06.359785 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a5d110a6-ebbc-4f71-889b-b6b562140ee7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:06.360202 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a5d110a6-ebbc-4f71-889b-b6b562140ee7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:06.380269 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-1c15444e-6d16-48a2-951a-852e19471bb7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:06.380877 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-1c15444e-6d16-48a2-951a-852e19471bb7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:06.388066 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1c15444e-6d16-48a2-951a-852e19471bb7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:06.388549 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1c15444e-6d16-48a2-951a-852e19471bb7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:06.388983 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1c15444e-6d16-48a2-951a-852e19471bb7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:06.389707 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1c15444e-6d16-48a2-951a-852e19471bb7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:06.390156 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1c15444e-6d16-48a2-951a-852e19471bb7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:06.391114 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1c15444e-6d16-48a2-951a-852e19471bb7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:06.421938 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-a5d110a6-ebbc-4f71-889b-b6b562140ee7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:06.430759 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-a5d110a6-ebbc-4f71-889b-b6b562140ee7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:06.432201 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-a5d110a6-ebbc-4f71-889b-b6b562140ee7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:06.433320 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-a5d110a6-ebbc-4f71-889b-b6b562140ee7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1338 microversion: 2.20 time: 0.086756 Sep 04 01:10:06.433870 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 435/848] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:10:06 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1338 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:10:06.520720 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-1c15444e-6d16-48a2-951a-852e19471bb7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:06.528806 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-1c15444e-6d16-48a2-951a-852e19471bb7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1670 microversion: 2.1 time: 0.152405 Sep 04 01:10:06.529393 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 414/849] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:06 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1670 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:10:07.461130 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-012a8059-4c31-4206-91b2-4a1abfe8b8c9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:07.461130 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-012a8059-4c31-4206-91b2-4a1abfe8b8c9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:07.467776 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-012a8059-4c31-4206-91b2-4a1abfe8b8c9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:07.467883 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-012a8059-4c31-4206-91b2-4a1abfe8b8c9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:07.468389 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-012a8059-4c31-4206-91b2-4a1abfe8b8c9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:07.469188 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-012a8059-4c31-4206-91b2-4a1abfe8b8c9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:07.469708 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-012a8059-4c31-4206-91b2-4a1abfe8b8c9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:07.470202 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-012a8059-4c31-4206-91b2-4a1abfe8b8c9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:07.531845 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-012a8059-4c31-4206-91b2-4a1abfe8b8c9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:07.540550 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-012a8059-4c31-4206-91b2-4a1abfe8b8c9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:07.542027 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-012a8059-4c31-4206-91b2-4a1abfe8b8c9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:07.543116 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-012a8059-4c31-4206-91b2-4a1abfe8b8c9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1338 microversion: 2.20 time: 0.087905 Sep 04 01:10:07.543767 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 436/850] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:10:07 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1338 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:10:07.553178 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-dc1afd28-4b66-4ad1-aefc-9d48abe1d073 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:07.553788 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-dc1afd28-4b66-4ad1-aefc-9d48abe1d073 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:07.561106 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-dc1afd28-4b66-4ad1-aefc-9d48abe1d073 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:07.561600 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-dc1afd28-4b66-4ad1-aefc-9d48abe1d073 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:07.562032 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-dc1afd28-4b66-4ad1-aefc-9d48abe1d073 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:07.562720 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-dc1afd28-4b66-4ad1-aefc-9d48abe1d073 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:07.563168 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-dc1afd28-4b66-4ad1-aefc-9d48abe1d073 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:07.563588 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-dc1afd28-4b66-4ad1-aefc-9d48abe1d073 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:07.861294 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-dc1afd28-4b66-4ad1-aefc-9d48abe1d073 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:07.870875 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-dc1afd28-4b66-4ad1-aefc-9d48abe1d073 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1670 microversion: 2.1 time: 0.321643 Sep 04 01:10:07.871468 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 415/851] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:07 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1670 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:08.569603 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-c3a0d49e-b4f1-4e13-8cef-60633c383164 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:08.571608 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-c3a0d49e-b4f1-4e13-8cef-60633c383164 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:08.578062 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c3a0d49e-b4f1-4e13-8cef-60633c383164 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:08.578545 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c3a0d49e-b4f1-4e13-8cef-60633c383164 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:08.579082 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c3a0d49e-b4f1-4e13-8cef-60633c383164 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:08.586945 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c3a0d49e-b4f1-4e13-8cef-60633c383164 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:08.586945 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c3a0d49e-b4f1-4e13-8cef-60633c383164 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:08.586945 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c3a0d49e-b4f1-4e13-8cef-60633c383164 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:08.798459 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-c3a0d49e-b4f1-4e13-8cef-60633c383164 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:08.810564 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-c3a0d49e-b4f1-4e13-8cef-60633c383164 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:08.812380 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-c3a0d49e-b4f1-4e13-8cef-60633c383164 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:08.813842 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-c3a0d49e-b4f1-4e13-8cef-60633c383164 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1405 microversion: 2.20 time: 0.248687 Sep 04 01:10:08.829903 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 437/852] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:10:08 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1405 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:10:08.895842 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-77738998-3c8a-449d-b1b0-d6c504ce90bc tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:08.897092 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-77738998-3c8a-449d-b1b0-d6c504ce90bc tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:08.904591 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-77738998-3c8a-449d-b1b0-d6c504ce90bc tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:08.905196 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-77738998-3c8a-449d-b1b0-d6c504ce90bc tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:08.905892 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-77738998-3c8a-449d-b1b0-d6c504ce90bc tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:08.907059 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-77738998-3c8a-449d-b1b0-d6c504ce90bc tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:08.907732 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-77738998-3c8a-449d-b1b0-d6c504ce90bc tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:08.908357 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-77738998-3c8a-449d-b1b0-d6c504ce90bc tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:08.981541 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-77738998-3c8a-449d-b1b0-d6c504ce90bc tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:08.995740 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-77738998-3c8a-449d-b1b0-d6c504ce90bc tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1604 microversion: 2.1 time: 0.103975 Sep 04 01:10:09.010461 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 416/853] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:08 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1604 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:09.023502 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-34a765db-89d3-4b85-983b-4330e4ebe8b5 req-c4756795-d35e-42be-8a09-69403490b7df service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "da1d8cad-5ea0-48fc-a5af-81eec3d1e94b", "tag": "19f203f1-14db-4bb0-aaad-c8e57c75d249"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:10:09.039840 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-34a765db-89d3-4b85-983b-4330e4ebe8b5 req-c4756795-d35e-42be-8a09-69403490b7df service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:09.040661 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-34a765db-89d3-4b85-983b-4330e4ebe8b5 req-c4756795-d35e-42be-8a09-69403490b7df service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:09.041278 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-34a765db-89d3-4b85-983b-4330e4ebe8b5 req-c4756795-d35e-42be-8a09-69403490b7df service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:09.057923 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-34a765db-89d3-4b85-983b-4330e4ebe8b5 req-c4756795-d35e-42be-8a09-69403490b7df service nova] Creating event network-changed:19f203f1-14db-4bb0-aaad-c8e57c75d249 for instance da1d8cad-5ea0-48fc-a5af-81eec3d1e94b on hv-cinder-maste Sep 04 01:10:09.065472 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-34a765db-89d3-4b85-983b-4330e4ebe8b5 req-c4756795-d35e-42be-8a09-69403490b7df service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.054727 Sep 04 01:10:09.066721 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 438/854] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:10:09 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:10:09.225007 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-3ff444ef-1646-48c1-94bb-4b351c9c5ddc req-69882341-3837-4bfa-bae3-f3c0ed44ff9d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d296b21a-c705-4d0b-8402-19a9c6f2e3d2", "name": "network-vif-deleted", "tag": "e9c83c9b-f295-4069-84cf-ff7c294ae4cd"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:10:09.235945 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-3ff444ef-1646-48c1-94bb-4b351c9c5ddc req-69882341-3837-4bfa-bae3-f3c0ed44ff9d service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:09.236402 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-3ff444ef-1646-48c1-94bb-4b351c9c5ddc req-69882341-3837-4bfa-bae3-f3c0ed44ff9d service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:09.236836 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-3ff444ef-1646-48c1-94bb-4b351c9c5ddc req-69882341-3837-4bfa-bae3-f3c0ed44ff9d service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:09.250191 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-3ff444ef-1646-48c1-94bb-4b351c9c5ddc req-69882341-3837-4bfa-bae3-f3c0ed44ff9d service nova] Creating event network-vif-deleted:e9c83c9b-f295-4069-84cf-ff7c294ae4cd for instance d296b21a-c705-4d0b-8402-19a9c6f2e3d2 on hv-cinder-maste Sep 04 01:10:09.256073 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-3ff444ef-1646-48c1-94bb-4b351c9c5ddc req-69882341-3837-4bfa-bae3-f3c0ed44ff9d service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034658 Sep 04 01:10:09.256900 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 417/855] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:10:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:10:09.842286 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-d85fcd82-b197-4f08-8942-9d638b0ee11c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:09.843789 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-d85fcd82-b197-4f08-8942-9d638b0ee11c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:09.851384 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d85fcd82-b197-4f08-8942-9d638b0ee11c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:09.851868 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d85fcd82-b197-4f08-8942-9d638b0ee11c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:09.852461 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d85fcd82-b197-4f08-8942-9d638b0ee11c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:09.853178 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d85fcd82-b197-4f08-8942-9d638b0ee11c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:09.853792 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d85fcd82-b197-4f08-8942-9d638b0ee11c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:09.854362 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d85fcd82-b197-4f08-8942-9d638b0ee11c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:09.999945 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-d85fcd82-b197-4f08-8942-9d638b0ee11c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:10.010829 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-d85fcd82-b197-4f08-8942-9d638b0ee11c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:10.012477 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-d85fcd82-b197-4f08-8942-9d638b0ee11c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:10.013686 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-d85fcd82-b197-4f08-8942-9d638b0ee11c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1405 microversion: 2.20 time: 0.177247 Sep 04 01:10:10.014244 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 439/856] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:10:09 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1405 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:10:10.026565 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-b0653966-9f3f-4408-adcb-cf087403aef8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:10.027837 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-b0653966-9f3f-4408-adcb-cf087403aef8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:10.035030 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b0653966-9f3f-4408-adcb-cf087403aef8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:10.035506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b0653966-9f3f-4408-adcb-cf087403aef8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:10.036113 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b0653966-9f3f-4408-adcb-cf087403aef8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:10.036929 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b0653966-9f3f-4408-adcb-cf087403aef8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:10.037415 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b0653966-9f3f-4408-adcb-cf087403aef8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:10.037865 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b0653966-9f3f-4408-adcb-cf087403aef8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:10.070636 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b0653966-9f3f-4408-adcb-cf087403aef8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:10.071119 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b0653966-9f3f-4408-adcb-cf087403aef8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:10.071545 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b0653966-9f3f-4408-adcb-cf087403aef8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:10.071986 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.objects.instance [None req-b0653966-9f3f-4408-adcb-cf087403aef8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lazy-loading 'fault' on Instance uuid d296b21a-c705-4d0b-8402-19a9c6f2e3d2 {{(pid=121037) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:10:10.115222 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-b0653966-9f3f-4408-adcb-cf087403aef8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:10.124024 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-b0653966-9f3f-4408-adcb-cf087403aef8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 200 len: 1326 microversion: 2.1 time: 0.103165 Sep 04 01:10:10.124740 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 418/857] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:10 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 1326 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:10:11.039094 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-10cb9617-09e9-4a98-ab6d-a9703f4f8417 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:11.039702 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-10cb9617-09e9-4a98-ab6d-a9703f4f8417 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:11.046192 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-10cb9617-09e9-4a98-ab6d-a9703f4f8417 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:11.046700 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-10cb9617-09e9-4a98-ab6d-a9703f4f8417 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:11.047135 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-10cb9617-09e9-4a98-ab6d-a9703f4f8417 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:11.047822 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-10cb9617-09e9-4a98-ab6d-a9703f4f8417 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:11.048272 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-10cb9617-09e9-4a98-ab6d-a9703f4f8417 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:11.048694 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-10cb9617-09e9-4a98-ab6d-a9703f4f8417 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:11.149416 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-eaa9760a-a9f1-425e-892f-253dacc48468 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:11.150030 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-eaa9760a-a9f1-425e-892f-253dacc48468 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:11.156436 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-eaa9760a-a9f1-425e-892f-253dacc48468 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:11.156921 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-eaa9760a-a9f1-425e-892f-253dacc48468 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:11.157435 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-eaa9760a-a9f1-425e-892f-253dacc48468 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:11.158132 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-eaa9760a-a9f1-425e-892f-253dacc48468 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:11.158582 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-eaa9760a-a9f1-425e-892f-253dacc48468 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:11.159211 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-eaa9760a-a9f1-425e-892f-253dacc48468 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:11.173237 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.wsgi [None req-eaa9760a-a9f1-425e-892f-253dacc48468 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] HTTP exception thrown: Instance d296b21a-c705-4d0b-8402-19a9c6f2e3d2 could not be found. Sep 04 01:10:11.174871 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-eaa9760a-a9f1-425e-892f-253dacc48468 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Request method failure captured: Sep 04 01:10:11.174871 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: request: GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 HTTP/1.1 Sep 04 01:10:11.174871 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept: application/json Sep 04 01:10:11.174871 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept-Encoding: identity Sep 04 01:10:11.174871 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Connection: close Sep 04 01:10:11.174871 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Length: 0 Sep 04 01:10:11.174871 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Type: application/json Sep 04 01:10:11.174871 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Host: 10.222.0.42 Sep 04 01:10:11.174871 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Openstack-System-Scope: None Sep 04 01:10:11.174871 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: User-Agent: python-urllib3/1.26.12 Sep 04 01:10:11.174871 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Auth-Token: gAAAAABjE_nJhkGHWb7T6kbA22g6MQpQUnzziOpw0WCnFjRF-qzwvonIMqa9XQHefbSvkVL4GGxkzosDVGSxep3ShpT_NSR1Y0CpdR-zYyq1zOcQJ4WQ7o2EXz_hogwf8tYG4DUEXAu4uYN44FQubcx8a7LGqRtbPamZnr7PotOUcFaQv_B5v7E Sep 04 01:10:11.174871 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Id: None Sep 04 01:10:11.174871 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Name: None Sep 04 01:10:11.174871 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Identity-Status: Confirmed Sep 04 01:10:11.174871 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Is-Admin-Project: True Sep 04 01:10:11.174871 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Id: default Sep 04 01:10:11.174871 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Name: Default Sep 04 01:10:11.174871 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Id: e7dca68418a14d7da9aa63ac25f202e8 Sep 04 01:10:11.174871 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Name: tempest-AttachVolumeTestJSON-1676343226 Sep 04 01:10:11.174871 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Role: reader,member Sep 04 01:10:11.174871 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Roles: reader,member Sep 04 01:10:11.176523 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/e7dca68418a14d7da9aa63ac25f202e8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_e7dca68418a14d7da9aa63ac25f202e8", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/e7dca68418a14d7da9aa63ac25f202e8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/e7dca68418a14d7da9aa63ac25f202e8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:10:11.176523 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant: tempest-AttachVolumeTestJSON-1676343226 Sep 04 01:10:11.176523 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Id: e7dca68418a14d7da9aa63ac25f202e8 Sep 04 01:10:11.176523 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1676343226 Sep 04 01:10:11.176523 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User: tempest-AttachVolumeTestJSON-1676343226-project Sep 04 01:10:11.176523 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Id: default Sep 04 01:10:11.176523 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Name: Default Sep 04 01:10:11.176523 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Id: 44993cd93360442bb0afb28778ee83bc Sep 04 01:10:11.176523 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Name: tempest-AttachVolumeTestJSON-1676343226-project Sep 04 01:10:11.176523 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: method: > Sep 04 01:10:11.176523 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: exception: Instance d296b21a-c705-4d0b-8402-19a9c6f2e3d2 could not be found. Sep 04 01:10:11.176523 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: action_args: {'id': 'd296b21a-c705-4d0b-8402-19a9c6f2e3d2'} Sep 04 01:10:11.177791 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:10:11.177791 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:10:11.177791 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:10:11.177791 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:10:11.177791 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:10:11.177791 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:10:11.177791 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:10:11.177791 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:10:11.177791 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:10:11.177791 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:10:11.177791 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d296b21a-c705-4d0b-8402-19a9c6f2e3d2 could not be found. Sep 04 01:10:11.177791 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:10:11.177791 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:10:11.177791 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:10:11.177791 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:10:11.177791 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:10:11.177791 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:10:11.177791 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:10:11.177791 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:10:11.177791 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:10:11.179986 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:10:11.179986 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 04 01:10:11.179986 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 04 01:10:11.179986 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:10:11.179986 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:10:11.179986 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:10:11.179986 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:10:11.179986 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d296b21a-c705-4d0b-8402-19a9c6f2e3d2 could not be found. Sep 04 01:10:11.179986 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:10:11.179986 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:10:11.179986 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:10:11.179986 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:10:11.179986 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:10:11.179986 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:10:11.179986 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:10:11.179986 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:10:11.179986 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d296b21a-c705-4d0b-8402-19a9c6f2e3d2 could not be found. Sep 04 01:10:11.179986 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:10:11.182899 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-eaa9760a-a9f1-425e-892f-253dacc48468 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Returning 404 to user: Instance d296b21a-c705-4d0b-8402-19a9c6f2e3d2 could not be found. {{(pid=121037) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:10:11.182899 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-eaa9760a-a9f1-425e-892f-253dacc48468 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 404 len: 111 microversion: 2.1 time: 0.031392 Sep 04 01:10:11.182899 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 419/858] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:11 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 04 01:10:11.183334 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-10cb9617-09e9-4a98-ab6d-a9703f4f8417 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:11.185420 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-90d6379a-cf80-4c24-b2fb-0a00b3d57842 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '.version_select at 0x7fc10c50a4c0>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:11.190584 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-10cb9617-09e9-4a98-ab6d-a9703f4f8417 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:11.192166 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-10cb9617-09e9-4a98-ab6d-a9703f4f8417 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:11.193302 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-10cb9617-09e9-4a98-ab6d-a9703f4f8417 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1405 microversion: 2.20 time: 0.158456 Sep 04 01:10:11.193862 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 440/859] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:10:11 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1405 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:10:11.260790 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-90d6379a-cf80-4c24-b2fb-0a00b3d57842 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-677601389" status: 202 len: 0 microversion: 2.1 time: 0.078932 Sep 04 01:10:11.261398 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 420/860] 10.222.0.42 () {58 vars in 1218 bytes} [Sun Sep 4 01:10:11 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-677601389 => generated 0 bytes in 80 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 04 01:10:12.023572 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-2b51e09b-e498-432f-b1cc-051bb300e9d4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Action: 'create', calling method: .version_select at 0x7fc10c447af0>, body: {"keypair": {"name": "tempest-keypair-1390926425", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKHvrpSjn32PJJuSGH0bXAxqF1OGy3fiH+lIChfAEBd747Ui7TYwWmrPIw+LvZK2m/E4Gz1/GM5a7Fq5FxUFCqvqn8kCugQ1J8RYknzRWMG6NfuLlJJ8oIEwOVLtUbK9oQ=="}} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:10:12.029248 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell0:SELECT=6 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:10:12.041899 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-2b51e09b-e498-432f-b1cc-051bb300e9d4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Getting quotas for project e7dca68418a14d7da9aa63ac25f202e8. Resources: {'key_pairs'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 04 01:10:12.045844 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-2b51e09b-e498-432f-b1cc-051bb300e9d4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Getting quotas for user 44993cd93360442bb0afb28778ee83bc and project e7dca68418a14d7da9aa63ac25f202e8. Resources: {'key_pairs'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 04 01:10:12.062245 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_api:SELECT=93,nova_api:INSERT=5,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:10:12.124394 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-2b51e09b-e498-432f-b1cc-051bb300e9d4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.105294 Sep 04 01:10:12.124960 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 441/861] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Sep 4 01:10:12 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:10:12.214536 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell0:SELECT=2 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:10:12.215471 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell0:SELECT=6 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:10:12.220065 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-6dfd1e9c-e454-4eea-bf3b-97bf0069b4e4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:12.220679 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-6dfd1e9c-e454-4eea-bf3b-97bf0069b4e4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:12.228282 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6dfd1e9c-e454-4eea-bf3b-97bf0069b4e4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:12.233007 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6dfd1e9c-e454-4eea-bf3b-97bf0069b4e4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:12.233007 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6dfd1e9c-e454-4eea-bf3b-97bf0069b4e4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:12.233007 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6dfd1e9c-e454-4eea-bf3b-97bf0069b4e4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:12.233007 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6dfd1e9c-e454-4eea-bf3b-97bf0069b4e4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:12.233397 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6dfd1e9c-e454-4eea-bf3b-97bf0069b4e4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:12.386344 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-6dfd1e9c-e454-4eea-bf3b-97bf0069b4e4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:12.394679 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-6dfd1e9c-e454-4eea-bf3b-97bf0069b4e4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:12.396210 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-6dfd1e9c-e454-4eea-bf3b-97bf0069b4e4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:12.397303 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-6dfd1e9c-e454-4eea-bf3b-97bf0069b4e4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1405 microversion: 2.20 time: 0.183561 Sep 04 01:10:12.398413 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 421/862] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:10:12 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1405 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:10:13.422961 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-1f2b16f1-7ccb-43e8-8072-9c5445776ae3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:13.423592 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-1f2b16f1-7ccb-43e8-8072-9c5445776ae3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:13.431650 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1f2b16f1-7ccb-43e8-8072-9c5445776ae3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:13.432076 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1f2b16f1-7ccb-43e8-8072-9c5445776ae3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:13.432505 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1f2b16f1-7ccb-43e8-8072-9c5445776ae3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:13.433239 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1f2b16f1-7ccb-43e8-8072-9c5445776ae3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:13.433886 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1f2b16f1-7ccb-43e8-8072-9c5445776ae3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:13.434456 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1f2b16f1-7ccb-43e8-8072-9c5445776ae3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:13.573733 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-1f2b16f1-7ccb-43e8-8072-9c5445776ae3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:13.582165 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-1f2b16f1-7ccb-43e8-8072-9c5445776ae3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:13.583674 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-1f2b16f1-7ccb-43e8-8072-9c5445776ae3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:13.584779 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-1f2b16f1-7ccb-43e8-8072-9c5445776ae3 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1405 microversion: 2.20 time: 0.166764 Sep 04 01:10:13.585368 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 442/863] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:10:13 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1405 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:10:14.611631 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-5624ff15-a9db-44aa-a958-8a35d1c98320 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:14.612708 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-5624ff15-a9db-44aa-a958-8a35d1c98320 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:14.620195 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5624ff15-a9db-44aa-a958-8a35d1c98320 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:14.622795 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5624ff15-a9db-44aa-a958-8a35d1c98320 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:14.622795 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5624ff15-a9db-44aa-a958-8a35d1c98320 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:14.622795 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5624ff15-a9db-44aa-a958-8a35d1c98320 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:14.622795 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5624ff15-a9db-44aa-a958-8a35d1c98320 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:14.623350 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5624ff15-a9db-44aa-a958-8a35d1c98320 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:14.771888 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-5624ff15-a9db-44aa-a958-8a35d1c98320 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:14.771888 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=100 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:10:14.777670 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-5624ff15-a9db-44aa-a958-8a35d1c98320 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:14.779399 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-5624ff15-a9db-44aa-a958-8a35d1c98320 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:14.780828 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-5624ff15-a9db-44aa-a958-8a35d1c98320 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1405 microversion: 2.20 time: 0.172851 Sep 04 01:10:14.781139 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 422/864] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:10:14 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1405 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:10:14.896160 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-198ec49f-f568-4d4b-8409-631820b9e991 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-797198069", "imageRef": "a11315c3-ce81-478b-91e4-5353f189dfa7", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "1fb5aed8-696d-4f87-8b2a-7d29744538a8"}], "security_groups": [{"name": "tempest-securitygroup--449855155"}], "key_name": "tempest-keypair-1390926425", "user_data": "}} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:10:15.003947 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.network.neutron [None req-198ec49f-f568-4d4b-8409-631820b9e991 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] validate_networks() for [('1fb5aed8-696d-4f87-8b2a-7d29744538a8', None, None, None, None, None)] {{(pid=121036) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 04 01:10:15.380537 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-198ec49f-f568-4d4b-8409-631820b9e991 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Flavor limits 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 04 01:10:15.380962 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-198ec49f-f568-4d4b-8409-631820b9e991 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Image limits 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 04 01:10:15.381491 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-198ec49f-f568-4d4b-8409-631820b9e991 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Flavor pref 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 04 01:10:15.381906 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-198ec49f-f568-4d4b-8409-631820b9e991 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Image pref 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 04 01:10:15.382963 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-198ec49f-f568-4d4b-8409-631820b9e991 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 04 01:10:15.391712 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-198ec49f-f568-4d4b-8409-631820b9e991 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:15.392192 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-198ec49f-f568-4d4b-8409-631820b9e991 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:15.392629 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-198ec49f-f568-4d4b-8409-631820b9e991 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:15.393394 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-198ec49f-f568-4d4b-8409-631820b9e991 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:15.393846 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-198ec49f-f568-4d4b-8409-631820b9e991 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:15.394272 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-198ec49f-f568-4d4b-8409-631820b9e991 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:15.410622 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-198ec49f-f568-4d4b-8409-631820b9e991 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Getting quotas for project e7dca68418a14d7da9aa63ac25f202e8. Resources: {'ram', 'instances', 'cores'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 04 01:10:15.414802 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-198ec49f-f568-4d4b-8409-631820b9e991 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Getting quotas for user 44993cd93360442bb0afb28778ee83bc and project e7dca68418a14d7da9aa63ac25f202e8. Resources: {'ram', 'instances', 'cores'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 04 01:10:15.425684 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-198ec49f-f568-4d4b-8409-631820b9e991 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Going to run 1 instances... {{(pid=121036) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 04 01:10:15.428000 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-198ec49f-f568-4d4b-8409-631820b9e991 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a11315c3-ce81-478b-91e4-5353f189dfa7',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121036) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 04 01:10:15.548881 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-198ec49f-f568-4d4b-8409-631820b9e991 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.774560 Sep 04 01:10:15.556773 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 443/865] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 4 01:10:14 2022] POST /compute/v2.1/servers => generated 401 bytes in 776 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 04 01:10:15.562132 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-29f24742-0894-4cef-aa2f-ef38c569c8a3 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:15.562856 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-29f24742-0894-4cef-aa2f-ef38c569c8a3 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:15.566316 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_api:SELECT=95,nova_api:UPDATE=1,nova_api:INSERT=3,nova_api:DELETE=1 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:10:15.625921 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-29f24742-0894-4cef-aa2f-ef38c569c8a3 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:15.633731 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-29f24742-0894-4cef-aa2f-ef38c569c8a3 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219" status: 200 len: 1237 microversion: 2.1 time: 0.075620 Sep 04 01:10:15.634559 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 423/866] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:15 2022] GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219 => generated 1237 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:15.907889 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-14989228-df6a-4e69-8ff9-a74067935aa8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:15.910765 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-14989228-df6a-4e69-8ff9-a74067935aa8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:15.915906 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-14989228-df6a-4e69-8ff9-a74067935aa8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:15.916388 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-14989228-df6a-4e69-8ff9-a74067935aa8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:15.916821 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-14989228-df6a-4e69-8ff9-a74067935aa8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:15.917585 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-14989228-df6a-4e69-8ff9-a74067935aa8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:15.918037 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-14989228-df6a-4e69-8ff9-a74067935aa8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:15.918464 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-14989228-df6a-4e69-8ff9-a74067935aa8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:15.933654 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:10:16.055340 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-14989228-df6a-4e69-8ff9-a74067935aa8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:16.070307 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-14989228-df6a-4e69-8ff9-a74067935aa8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:16.073429 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-14989228-df6a-4e69-8ff9-a74067935aa8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:16.075586 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-14989228-df6a-4e69-8ff9-a74067935aa8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1405 microversion: 2.20 time: 0.273213 Sep 04 01:10:16.076587 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 444/867] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:10:15 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1405 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:10:16.658404 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-a2601190-e8a9-4bf7-bb19-c606af967a03 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:16.659046 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-a2601190-e8a9-4bf7-bb19-c606af967a03 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:16.668423 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a2601190-e8a9-4bf7-bb19-c606af967a03 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:16.668908 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a2601190-e8a9-4bf7-bb19-c606af967a03 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:16.669380 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a2601190-e8a9-4bf7-bb19-c606af967a03 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:16.670133 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a2601190-e8a9-4bf7-bb19-c606af967a03 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:16.670590 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a2601190-e8a9-4bf7-bb19-c606af967a03 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:16.671019 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a2601190-e8a9-4bf7-bb19-c606af967a03 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:16.732297 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-a2601190-e8a9-4bf7-bb19-c606af967a03 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:16.740300 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-a2601190-e8a9-4bf7-bb19-c606af967a03 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219" status: 200 len: 1241 microversion: 2.1 time: 0.086162 Sep 04 01:10:16.745170 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 424/868] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:16 2022] GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219 => generated 1241 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:17.097616 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-1cb2cb57-fb30-442f-b85d-967b934d59eb req-c17271f5-0b97-40d3-b1c1-db4ee59163df service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "da1d8cad-5ea0-48fc-a5af-81eec3d1e94b", "name": "network-vif-plugged", "status": "completed", "tag": "19f203f1-14db-4bb0-aaad-c8e57c75d249"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:10:17.103325 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-aeec1f01-7cb1-4b71-b2f7-366d547f4a2d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:17.104618 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-aeec1f01-7cb1-4b71-b2f7-366d547f4a2d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:17.111615 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-aeec1f01-7cb1-4b71-b2f7-366d547f4a2d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:17.112212 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-aeec1f01-7cb1-4b71-b2f7-366d547f4a2d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:17.112980 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-aeec1f01-7cb1-4b71-b2f7-366d547f4a2d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:17.113771 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-aeec1f01-7cb1-4b71-b2f7-366d547f4a2d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:17.114187 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-1cb2cb57-fb30-442f-b85d-967b934d59eb req-c17271f5-0b97-40d3-b1c1-db4ee59163df service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:17.114319 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-aeec1f01-7cb1-4b71-b2f7-366d547f4a2d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:17.114673 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-1cb2cb57-fb30-442f-b85d-967b934d59eb req-c17271f5-0b97-40d3-b1c1-db4ee59163df service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:17.114807 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-aeec1f01-7cb1-4b71-b2f7-366d547f4a2d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:17.115114 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-1cb2cb57-fb30-442f-b85d-967b934d59eb req-c17271f5-0b97-40d3-b1c1-db4ee59163df service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:17.127782 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-1cb2cb57-fb30-442f-b85d-967b934d59eb req-c17271f5-0b97-40d3-b1c1-db4ee59163df service nova] Creating event network-vif-plugged:19f203f1-14db-4bb0-aaad-c8e57c75d249 for instance da1d8cad-5ea0-48fc-a5af-81eec3d1e94b on hv-cinder-maste Sep 04 01:10:17.133190 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-1cb2cb57-fb30-442f-b85d-967b934d59eb req-c17271f5-0b97-40d3-b1c1-db4ee59163df service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038949 Sep 04 01:10:17.133785 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 445/869] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:10:17 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:10:17.257810 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-aeec1f01-7cb1-4b71-b2f7-366d547f4a2d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:17.267104 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-aeec1f01-7cb1-4b71-b2f7-366d547f4a2d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:17.268967 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-aeec1f01-7cb1-4b71-b2f7-366d547f4a2d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:17.270182 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-aeec1f01-7cb1-4b71-b2f7-366d547f4a2d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1405 microversion: 2.20 time: 0.172438 Sep 04 01:10:17.270864 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 425/870] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:10:17 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1405 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:10:17.765426 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-32703c6d-a2c7-417d-86e7-683c6668daf8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:17.768158 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-32703c6d-a2c7-417d-86e7-683c6668daf8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:17.772531 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-32703c6d-a2c7-417d-86e7-683c6668daf8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:17.773652 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-32703c6d-a2c7-417d-86e7-683c6668daf8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:17.774121 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-32703c6d-a2c7-417d-86e7-683c6668daf8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:17.774826 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-32703c6d-a2c7-417d-86e7-683c6668daf8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:17.775273 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-32703c6d-a2c7-417d-86e7-683c6668daf8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:17.775694 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-32703c6d-a2c7-417d-86e7-683c6668daf8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:17.843385 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-32703c6d-a2c7-417d-86e7-683c6668daf8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:17.851611 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-32703c6d-a2c7-417d-86e7-683c6668daf8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219" status: 200 len: 1297 microversion: 2.1 time: 0.089979 Sep 04 01:10:17.852186 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 446/871] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:17 2022] GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219 => generated 1297 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:18.294932 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-48dc2fa6-164b-45d8-87dd-003f559dfde4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:18.295546 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-48dc2fa6-164b-45d8-87dd-003f559dfde4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:18.302108 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-48dc2fa6-164b-45d8-87dd-003f559dfde4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:18.302588 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-48dc2fa6-164b-45d8-87dd-003f559dfde4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:18.303023 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-48dc2fa6-164b-45d8-87dd-003f559dfde4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:18.303706 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-48dc2fa6-164b-45d8-87dd-003f559dfde4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:18.304154 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-48dc2fa6-164b-45d8-87dd-003f559dfde4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:18.305127 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-48dc2fa6-164b-45d8-87dd-003f559dfde4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:18.435626 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-48dc2fa6-164b-45d8-87dd-003f559dfde4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:18.443702 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-48dc2fa6-164b-45d8-87dd-003f559dfde4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:18.445725 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-48dc2fa6-164b-45d8-87dd-003f559dfde4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:18.446821 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-48dc2fa6-164b-45d8-87dd-003f559dfde4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1405 microversion: 2.20 time: 0.155824 Sep 04 01:10:18.447379 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 426/872] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:10:18 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1405 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:10:18.875895 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-cac7bf6a-14f5-4394-8abb-3b28ee0482d5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:18.876587 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-cac7bf6a-14f5-4394-8abb-3b28ee0482d5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:18.885485 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cac7bf6a-14f5-4394-8abb-3b28ee0482d5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:18.885970 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cac7bf6a-14f5-4394-8abb-3b28ee0482d5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:18.886403 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cac7bf6a-14f5-4394-8abb-3b28ee0482d5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:18.887104 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cac7bf6a-14f5-4394-8abb-3b28ee0482d5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:18.887557 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cac7bf6a-14f5-4394-8abb-3b28ee0482d5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:18.887975 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cac7bf6a-14f5-4394-8abb-3b28ee0482d5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:18.952914 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-cac7bf6a-14f5-4394-8abb-3b28ee0482d5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:18.961610 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-cac7bf6a-14f5-4394-8abb-3b28ee0482d5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219" status: 200 len: 1295 microversion: 2.1 time: 0.089021 Sep 04 01:10:18.962258 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 447/873] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:18 2022] GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219 => generated 1295 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:19.471845 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-2a7af497-3e8f-420c-a9de-8f3698a35e17 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:19.472453 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-2a7af497-3e8f-420c-a9de-8f3698a35e17 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:19.479047 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2a7af497-3e8f-420c-a9de-8f3698a35e17 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:19.479524 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2a7af497-3e8f-420c-a9de-8f3698a35e17 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:19.479955 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2a7af497-3e8f-420c-a9de-8f3698a35e17 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:19.480636 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2a7af497-3e8f-420c-a9de-8f3698a35e17 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:19.481110 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2a7af497-3e8f-420c-a9de-8f3698a35e17 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:19.481530 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2a7af497-3e8f-420c-a9de-8f3698a35e17 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:19.673597 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-2a7af497-3e8f-420c-a9de-8f3698a35e17 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:19.681972 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-2a7af497-3e8f-420c-a9de-8f3698a35e17 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:19.683461 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-2a7af497-3e8f-420c-a9de-8f3698a35e17 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:19.684563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-2a7af497-3e8f-420c-a9de-8f3698a35e17 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1405 microversion: 2.20 time: 0.216683 Sep 04 01:10:19.685144 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 427/874] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:10:19 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1405 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:10:19.985914 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-d3a39a4d-1429-4020-bb18-5565caadc621 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:19.986524 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-d3a39a4d-1429-4020-bb18-5565caadc621 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:19.993820 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d3a39a4d-1429-4020-bb18-5565caadc621 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:19.994290 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d3a39a4d-1429-4020-bb18-5565caadc621 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:19.994722 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d3a39a4d-1429-4020-bb18-5565caadc621 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:19.995406 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d3a39a4d-1429-4020-bb18-5565caadc621 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:19.995852 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d3a39a4d-1429-4020-bb18-5565caadc621 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:19.996269 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d3a39a4d-1429-4020-bb18-5565caadc621 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:20.131807 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-d3a39a4d-1429-4020-bb18-5565caadc621 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:20.136967 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-d3a39a4d-1429-4020-bb18-5565caadc621 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219" status: 200 len: 1362 microversion: 2.1 time: 0.155108 Sep 04 01:10:20.137583 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 448/875] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:19 2022] GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219 => generated 1362 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:20.299605 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-acb99b57-a1a9-48fb-ae4b-466696cc6952 req-b0bb5fe7-bd69-4673-b58a-fa9cd88d9084 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "52505c46-cb03-49ec-a7ba-696d7c6ac219", "tag": "73f5bdf6-a4ae-435e-9050-14b4d54218f9"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:10:20.309840 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-acb99b57-a1a9-48fb-ae4b-466696cc6952 req-b0bb5fe7-bd69-4673-b58a-fa9cd88d9084 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:20.310312 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-acb99b57-a1a9-48fb-ae4b-466696cc6952 req-b0bb5fe7-bd69-4673-b58a-fa9cd88d9084 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:20.310742 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-acb99b57-a1a9-48fb-ae4b-466696cc6952 req-b0bb5fe7-bd69-4673-b58a-fa9cd88d9084 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:20.322714 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-acb99b57-a1a9-48fb-ae4b-466696cc6952 req-b0bb5fe7-bd69-4673-b58a-fa9cd88d9084 service nova] Creating event network-changed:73f5bdf6-a4ae-435e-9050-14b4d54218f9 for instance 52505c46-cb03-49ec-a7ba-696d7c6ac219 on hv-cinder-maste Sep 04 01:10:20.327776 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-acb99b57-a1a9-48fb-ae4b-466696cc6952 req-b0bb5fe7-bd69-4673-b58a-fa9cd88d9084 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031906 Sep 04 01:10:20.328448 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 428/876] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:10:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:10:20.709360 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-3d334ed2-6b53-4a53-b2ef-509c2ae466c6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:20.709976 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-3d334ed2-6b53-4a53-b2ef-509c2ae466c6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:20.716181 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3d334ed2-6b53-4a53-b2ef-509c2ae466c6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:20.716611 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3d334ed2-6b53-4a53-b2ef-509c2ae466c6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:20.717042 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3d334ed2-6b53-4a53-b2ef-509c2ae466c6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:20.717773 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3d334ed2-6b53-4a53-b2ef-509c2ae466c6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:20.718224 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3d334ed2-6b53-4a53-b2ef-509c2ae466c6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:20.718650 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3d334ed2-6b53-4a53-b2ef-509c2ae466c6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:20.848630 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-3d334ed2-6b53-4a53-b2ef-509c2ae466c6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:20.857772 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-3d334ed2-6b53-4a53-b2ef-509c2ae466c6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:20.859295 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-3d334ed2-6b53-4a53-b2ef-509c2ae466c6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:20.860466 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-3d334ed2-6b53-4a53-b2ef-509c2ae466c6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1597 microversion: 2.20 time: 0.155068 Sep 04 01:10:20.861137 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 449/877] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:10:20 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1597 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:10:20.887240 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-bee17b4f-9940-466e-8f34-b8fe8a8de124 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:20.887813 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-bee17b4f-9940-466e-8f34-b8fe8a8de124 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:20.894018 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-bee17b4f-9940-466e-8f34-b8fe8a8de124 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:20.894497 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-bee17b4f-9940-466e-8f34-b8fe8a8de124 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:20.894946 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-bee17b4f-9940-466e-8f34-b8fe8a8de124 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:20.895625 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-bee17b4f-9940-466e-8f34-b8fe8a8de124 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:20.896067 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-bee17b4f-9940-466e-8f34-b8fe8a8de124 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:20.896659 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-bee17b4f-9940-466e-8f34-b8fe8a8de124 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:21.014046 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-bee17b4f-9940-466e-8f34-b8fe8a8de124 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.131735 Sep 04 01:10:21.014647 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 429/878] 10.222.0.42 () {60 vars in 1316 bytes} [Sun Sep 4 01:10:20 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b/os-interface => generated 285 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 04 01:10:21.160856 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-24aca7ec-05b6-4055-9072-95e8c6cf1dcb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:21.161732 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-24aca7ec-05b6-4055-9072-95e8c6cf1dcb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:21.167847 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-24aca7ec-05b6-4055-9072-95e8c6cf1dcb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:21.168326 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-24aca7ec-05b6-4055-9072-95e8c6cf1dcb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:21.171686 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-24aca7ec-05b6-4055-9072-95e8c6cf1dcb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:21.171686 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-24aca7ec-05b6-4055-9072-95e8c6cf1dcb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:21.171686 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-24aca7ec-05b6-4055-9072-95e8c6cf1dcb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:21.171686 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-24aca7ec-05b6-4055-9072-95e8c6cf1dcb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:21.434282 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-24aca7ec-05b6-4055-9072-95e8c6cf1dcb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:21.441859 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-24aca7ec-05b6-4055-9072-95e8c6cf1dcb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219" status: 200 len: 1362 microversion: 2.1 time: 0.284973 Sep 04 01:10:21.442422 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 450/879] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:21 2022] GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219 => generated 1362 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:22.088845 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-ea1817ac-c7f9-41f5-b750-e17cab90ddc3 req-4ce57501-c60c-47c1-b98a-92c7a0824722 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "da1d8cad-5ea0-48fc-a5af-81eec3d1e94b", "tag": "19f203f1-14db-4bb0-aaad-c8e57c75d249"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:10:22.098672 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-ea1817ac-c7f9-41f5-b750-e17cab90ddc3 req-4ce57501-c60c-47c1-b98a-92c7a0824722 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:22.099146 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-ea1817ac-c7f9-41f5-b750-e17cab90ddc3 req-4ce57501-c60c-47c1-b98a-92c7a0824722 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:22.099583 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-ea1817ac-c7f9-41f5-b750-e17cab90ddc3 req-4ce57501-c60c-47c1-b98a-92c7a0824722 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:22.110862 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-ea1817ac-c7f9-41f5-b750-e17cab90ddc3 req-4ce57501-c60c-47c1-b98a-92c7a0824722 service nova] Creating event network-changed:19f203f1-14db-4bb0-aaad-c8e57c75d249 for instance da1d8cad-5ea0-48fc-a5af-81eec3d1e94b on hv-cinder-maste Sep 04 01:10:22.116554 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-ea1817ac-c7f9-41f5-b750-e17cab90ddc3 req-4ce57501-c60c-47c1-b98a-92c7a0824722 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032006 Sep 04 01:10:22.117630 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 430/880] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:10:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:10:22.470935 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-3b755882-e2ef-42f9-8fd7-bf96c33bd299 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:22.471781 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-3b755882-e2ef-42f9-8fd7-bf96c33bd299 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:22.478505 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3b755882-e2ef-42f9-8fd7-bf96c33bd299 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:22.478991 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3b755882-e2ef-42f9-8fd7-bf96c33bd299 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:22.479601 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3b755882-e2ef-42f9-8fd7-bf96c33bd299 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:22.480302 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3b755882-e2ef-42f9-8fd7-bf96c33bd299 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:22.484173 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3b755882-e2ef-42f9-8fd7-bf96c33bd299 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:22.484173 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3b755882-e2ef-42f9-8fd7-bf96c33bd299 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:22.622102 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-3b755882-e2ef-42f9-8fd7-bf96c33bd299 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:22.630365 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-3b755882-e2ef-42f9-8fd7-bf96c33bd299 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219" status: 200 len: 1362 microversion: 2.1 time: 0.166098 Sep 04 01:10:22.640154 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 451/881] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:22 2022] GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219 => generated 1362 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:22.873028 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-de1d766c-1464-470d-bf65-6639a11573bd tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:22.873642 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-de1d766c-1464-470d-bf65-6639a11573bd tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:22.879998 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-de1d766c-1464-470d-bf65-6639a11573bd tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:22.880545 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-de1d766c-1464-470d-bf65-6639a11573bd tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:22.880924 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-de1d766c-1464-470d-bf65-6639a11573bd tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:22.881652 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-de1d766c-1464-470d-bf65-6639a11573bd tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:22.882106 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-de1d766c-1464-470d-bf65-6639a11573bd tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:22.882594 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-de1d766c-1464-470d-bf65-6639a11573bd tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:22.972584 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.objects.instance [None req-de1d766c-1464-470d-bf65-6639a11573bd tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lazy-loading 'services' on Instance uuid 2570ee6c-f335-41bc-8e67-f89a62b28134 {{(pid=121037) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:10:23.254807 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-de1d766c-1464-470d-bf65-6639a11573bd tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "DELETE /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134/os-volume_attachments/10a678c8-9649-469a-b1ef-7e0530ce26b8" status: 202 len: 0 microversion: 2.1 time: 0.386058 Sep 04 01:10:23.255519 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 431/882] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Sep 4 01:10:22 2022] DELETE /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134/os-volume_attachments/10a678c8-9649-469a-b1ef-7e0530ce26b8 => generated 0 bytes in 389 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 04 01:10:23.264978 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-8ad04fbb-5b68-49f2-82d2-d6c7d95c6195 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:23.267564 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-8ad04fbb-5b68-49f2-82d2-d6c7d95c6195 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:23.277978 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8ad04fbb-5b68-49f2-82d2-d6c7d95c6195 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:23.278468 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8ad04fbb-5b68-49f2-82d2-d6c7d95c6195 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:23.283072 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8ad04fbb-5b68-49f2-82d2-d6c7d95c6195 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:23.283072 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8ad04fbb-5b68-49f2-82d2-d6c7d95c6195 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:23.283072 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8ad04fbb-5b68-49f2-82d2-d6c7d95c6195 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:23.283072 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8ad04fbb-5b68-49f2-82d2-d6c7d95c6195 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:23.313110 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-8ad04fbb-5b68-49f2-82d2-d6c7d95c6195 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052524 Sep 04 01:10:23.313709 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 452/883] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 4 01:10:23 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:10:23.656012 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-86bb4db9-f7be-42c2-9c16-d756e7459f03 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:23.656657 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-86bb4db9-f7be-42c2-9c16-d756e7459f03 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:23.663381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-86bb4db9-f7be-42c2-9c16-d756e7459f03 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:23.664498 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-86bb4db9-f7be-42c2-9c16-d756e7459f03 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:23.664980 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-86bb4db9-f7be-42c2-9c16-d756e7459f03 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:23.665704 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-86bb4db9-f7be-42c2-9c16-d756e7459f03 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:23.666161 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-86bb4db9-f7be-42c2-9c16-d756e7459f03 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:23.666591 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-86bb4db9-f7be-42c2-9c16-d756e7459f03 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:23.796765 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-86bb4db9-f7be-42c2-9c16-d756e7459f03 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:23.804512 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-86bb4db9-f7be-42c2-9c16-d756e7459f03 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219" status: 200 len: 1362 microversion: 2.1 time: 0.152939 Sep 04 01:10:23.805108 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 432/884] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:23 2022] GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219 => generated 1362 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:10:24.325975 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-2261078a-6a95-4269-9b97-cf7ff0e99f02 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:24.328177 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-2261078a-6a95-4269-9b97-cf7ff0e99f02 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:24.335053 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2261078a-6a95-4269-9b97-cf7ff0e99f02 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:24.335519 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2261078a-6a95-4269-9b97-cf7ff0e99f02 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:24.335951 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2261078a-6a95-4269-9b97-cf7ff0e99f02 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:24.336632 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2261078a-6a95-4269-9b97-cf7ff0e99f02 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:24.337320 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2261078a-6a95-4269-9b97-cf7ff0e99f02 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:24.337754 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2261078a-6a95-4269-9b97-cf7ff0e99f02 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:24.368208 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-2261078a-6a95-4269-9b97-cf7ff0e99f02 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046931 Sep 04 01:10:24.369320 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 453/885] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 4 01:10:24 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:10:24.827819 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-ea3c4475-34ef-4ad3-ba58-f5ae73bccd0d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:24.828433 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-ea3c4475-34ef-4ad3-ba58-f5ae73bccd0d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:24.834877 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea3c4475-34ef-4ad3-ba58-f5ae73bccd0d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:24.835354 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea3c4475-34ef-4ad3-ba58-f5ae73bccd0d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:24.835787 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea3c4475-34ef-4ad3-ba58-f5ae73bccd0d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:24.836471 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea3c4475-34ef-4ad3-ba58-f5ae73bccd0d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:24.836918 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea3c4475-34ef-4ad3-ba58-f5ae73bccd0d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:24.837988 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea3c4475-34ef-4ad3-ba58-f5ae73bccd0d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:24.965575 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-ea3c4475-34ef-4ad3-ba58-f5ae73bccd0d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:24.979903 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-ea3c4475-34ef-4ad3-ba58-f5ae73bccd0d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219" status: 200 len: 1362 microversion: 2.1 time: 0.155844 Sep 04 01:10:24.980576 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 433/886] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:24 2022] GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219 => generated 1362 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:25.382283 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-8ec8e7b1-0b28-4e0a-8faf-816c393831b1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:25.385037 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-8ec8e7b1-0b28-4e0a-8faf-816c393831b1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:25.391429 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8ec8e7b1-0b28-4e0a-8faf-816c393831b1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:25.391899 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8ec8e7b1-0b28-4e0a-8faf-816c393831b1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:25.392328 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8ec8e7b1-0b28-4e0a-8faf-816c393831b1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:25.393001 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8ec8e7b1-0b28-4e0a-8faf-816c393831b1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:25.393475 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8ec8e7b1-0b28-4e0a-8faf-816c393831b1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:25.393894 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8ec8e7b1-0b28-4e0a-8faf-816c393831b1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:25.409747 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell0:SELECT=2 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:10:25.421754 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell0:SELECT=10 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:10:25.426069 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-8ec8e7b1-0b28-4e0a-8faf-816c393831b1 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.047857 Sep 04 01:10:25.426628 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 454/887] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 4 01:10:25 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134/os-volume_attachments => generated 25 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 04 01:10:25.511308 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-c45a593d-017a-4a3c-a62c-652022501169 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:25.511799 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-c45a593d-017a-4a3c-a62c-652022501169 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:25.517757 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c45a593d-017a-4a3c-a62c-652022501169 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:25.518232 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c45a593d-017a-4a3c-a62c-652022501169 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:25.518662 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c45a593d-017a-4a3c-a62c-652022501169 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:25.519349 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c45a593d-017a-4a3c-a62c-652022501169 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:25.519791 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c45a593d-017a-4a3c-a62c-652022501169 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:25.520206 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c45a593d-017a-4a3c-a62c-652022501169 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:25.545346 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-c45a593d-017a-4a3c-a62c-652022501169 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Going to try to terminate instance {{(pid=121037) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 04 01:10:25.630749 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell0:SELECT=2 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:10:25.798044 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-c45a593d-017a-4a3c-a62c-652022501169 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "DELETE /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 204 len: 0 microversion: 2.1 time: 0.289938 Sep 04 01:10:25.798250 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 434/888] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 4 01:10:25 2022] DELETE /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 04 01:10:25.806514 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-7b7f518a-de48-469e-a93b-c1df3340021d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:25.807119 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-7b7f518a-de48-469e-a93b-c1df3340021d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:25.814072 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7b7f518a-de48-469e-a93b-c1df3340021d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:25.814555 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7b7f518a-de48-469e-a93b-c1df3340021d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:25.814991 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7b7f518a-de48-469e-a93b-c1df3340021d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:25.815713 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7b7f518a-de48-469e-a93b-c1df3340021d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:25.816164 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7b7f518a-de48-469e-a93b-c1df3340021d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:25.816589 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7b7f518a-de48-469e-a93b-c1df3340021d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:25.947836 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-7b7f518a-de48-469e-a93b-c1df3340021d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:25.955514 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-7b7f518a-de48-469e-a93b-c1df3340021d tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1689 microversion: 2.1 time: 0.152962 Sep 04 01:10:25.956085 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 455/889] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:25 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1689 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:26.003704 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-7ec8610c-e284-4fc9-8f85-6b956c538074 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:26.004420 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-7ec8610c-e284-4fc9-8f85-6b956c538074 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:26.011342 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7ec8610c-e284-4fc9-8f85-6b956c538074 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:26.011817 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7ec8610c-e284-4fc9-8f85-6b956c538074 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:26.012275 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7ec8610c-e284-4fc9-8f85-6b956c538074 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:26.012959 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7ec8610c-e284-4fc9-8f85-6b956c538074 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:26.013442 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7ec8610c-e284-4fc9-8f85-6b956c538074 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:26.013865 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7ec8610c-e284-4fc9-8f85-6b956c538074 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:26.141330 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-7ec8610c-e284-4fc9-8f85-6b956c538074 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:26.148955 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-7ec8610c-e284-4fc9-8f85-6b956c538074 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219" status: 200 len: 1362 microversion: 2.1 time: 0.149505 Sep 04 01:10:26.149578 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 435/890] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:25 2022] GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219 => generated 1362 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:26.979783 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-965d8bb5-4ea8-468e-bb53-ba753221601c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:26.980382 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-965d8bb5-4ea8-468e-bb53-ba753221601c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:26.987055 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-965d8bb5-4ea8-468e-bb53-ba753221601c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:26.987534 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-965d8bb5-4ea8-468e-bb53-ba753221601c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:26.988254 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-965d8bb5-4ea8-468e-bb53-ba753221601c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:26.988928 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-965d8bb5-4ea8-468e-bb53-ba753221601c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:26.989402 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-965d8bb5-4ea8-468e-bb53-ba753221601c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:26.989827 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-965d8bb5-4ea8-468e-bb53-ba753221601c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:27.124627 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-965d8bb5-4ea8-468e-bb53-ba753221601c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:27.132470 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-965d8bb5-4ea8-468e-bb53-ba753221601c tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1689 microversion: 2.1 time: 0.157030 Sep 04 01:10:27.133169 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 456/891] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:26 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1689 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:27.172683 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-893c5286-e3da-4d56-937b-190f1df484a1 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:27.173774 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-893c5286-e3da-4d56-937b-190f1df484a1 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:27.184701 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-893c5286-e3da-4d56-937b-190f1df484a1 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:27.185455 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-893c5286-e3da-4d56-937b-190f1df484a1 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:27.185598 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-5dbf4169-48b1-4c79-b77c-8e0302798edf req-734d383c-95f9-4217-8199-d583ba420548 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "52505c46-cb03-49ec-a7ba-696d7c6ac219", "name": "network-vif-plugged", "status": "completed", "tag": "73f5bdf6-a4ae-435e-9050-14b4d54218f9"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:10:27.186258 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-893c5286-e3da-4d56-937b-190f1df484a1 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:27.187021 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-893c5286-e3da-4d56-937b-190f1df484a1 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:27.187482 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-893c5286-e3da-4d56-937b-190f1df484a1 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:27.187913 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-893c5286-e3da-4d56-937b-190f1df484a1 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:27.195866 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-5dbf4169-48b1-4c79-b77c-8e0302798edf req-734d383c-95f9-4217-8199-d583ba420548 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:27.196360 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-5dbf4169-48b1-4c79-b77c-8e0302798edf req-734d383c-95f9-4217-8199-d583ba420548 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:27.196982 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-5dbf4169-48b1-4c79-b77c-8e0302798edf req-734d383c-95f9-4217-8199-d583ba420548 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:27.208369 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-5dbf4169-48b1-4c79-b77c-8e0302798edf req-734d383c-95f9-4217-8199-d583ba420548 service nova] Creating event network-vif-plugged:73f5bdf6-a4ae-435e-9050-14b4d54218f9 for instance 52505c46-cb03-49ec-a7ba-696d7c6ac219 on hv-cinder-maste Sep 04 01:10:27.214331 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-5dbf4169-48b1-4c79-b77c-8e0302798edf req-734d383c-95f9-4217-8199-d583ba420548 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033094 Sep 04 01:10:27.215051 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 457/892] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:10:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:10:27.327650 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-893c5286-e3da-4d56-937b-190f1df484a1 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:27.335336 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-893c5286-e3da-4d56-937b-190f1df484a1 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219" status: 200 len: 1362 microversion: 2.1 time: 0.166856 Sep 04 01:10:27.335901 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 436/893] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:27 2022] GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219 => generated 1362 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:28.156860 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-986bfec3-fc59-4726-9363-c4583f0c946a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:28.157583 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-986bfec3-fc59-4726-9363-c4583f0c946a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:28.163827 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-986bfec3-fc59-4726-9363-c4583f0c946a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:28.164307 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-986bfec3-fc59-4726-9363-c4583f0c946a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:28.164743 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-986bfec3-fc59-4726-9363-c4583f0c946a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:28.165965 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-986bfec3-fc59-4726-9363-c4583f0c946a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:28.166419 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-986bfec3-fc59-4726-9363-c4583f0c946a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:28.166850 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-986bfec3-fc59-4726-9363-c4583f0c946a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:28.294004 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-986bfec3-fc59-4726-9363-c4583f0c946a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:28.301712 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-986bfec3-fc59-4726-9363-c4583f0c946a tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1689 microversion: 2.1 time: 0.149598 Sep 04 01:10:28.302274 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 458/894] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:28 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1689 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:28.359259 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-486bc8ae-f714-43b5-a763-0b7b4e481ef4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:28.359864 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-486bc8ae-f714-43b5-a763-0b7b4e481ef4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:28.366455 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-486bc8ae-f714-43b5-a763-0b7b4e481ef4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:28.366920 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-486bc8ae-f714-43b5-a763-0b7b4e481ef4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:28.367341 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-486bc8ae-f714-43b5-a763-0b7b4e481ef4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:28.368013 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-486bc8ae-f714-43b5-a763-0b7b4e481ef4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:28.369233 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-486bc8ae-f714-43b5-a763-0b7b4e481ef4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:28.369654 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-486bc8ae-f714-43b5-a763-0b7b4e481ef4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:28.495629 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-486bc8ae-f714-43b5-a763-0b7b4e481ef4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:28.503469 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-486bc8ae-f714-43b5-a763-0b7b4e481ef4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219" status: 200 len: 1362 microversion: 2.1 time: 0.148117 Sep 04 01:10:28.504025 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 437/895] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:28 2022] GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219 => generated 1362 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:29.325213 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-99e18b0b-a752-4e9c-8c2e-77a4910910ca tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:29.325813 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-99e18b0b-a752-4e9c-8c2e-77a4910910ca tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:29.332367 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-99e18b0b-a752-4e9c-8c2e-77a4910910ca tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:29.332879 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-99e18b0b-a752-4e9c-8c2e-77a4910910ca tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:29.333356 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-99e18b0b-a752-4e9c-8c2e-77a4910910ca tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:29.334047 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-99e18b0b-a752-4e9c-8c2e-77a4910910ca tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:29.334501 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-99e18b0b-a752-4e9c-8c2e-77a4910910ca tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:29.334925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-99e18b0b-a752-4e9c-8c2e-77a4910910ca tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:29.470154 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-99e18b0b-a752-4e9c-8c2e-77a4910910ca tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:29.478741 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-99e18b0b-a752-4e9c-8c2e-77a4910910ca tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1689 microversion: 2.1 time: 0.157386 Sep 04 01:10:29.479538 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 459/896] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:29 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1689 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:29.528237 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-7016d8f2-4a47-463a-93c0-cde04dbc04b7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:29.528921 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-7016d8f2-4a47-463a-93c0-cde04dbc04b7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:29.535298 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7016d8f2-4a47-463a-93c0-cde04dbc04b7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:29.535782 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7016d8f2-4a47-463a-93c0-cde04dbc04b7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:29.536215 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7016d8f2-4a47-463a-93c0-cde04dbc04b7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:29.536912 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7016d8f2-4a47-463a-93c0-cde04dbc04b7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:29.537393 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7016d8f2-4a47-463a-93c0-cde04dbc04b7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:29.537824 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7016d8f2-4a47-463a-93c0-cde04dbc04b7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:29.713678 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-7016d8f2-4a47-463a-93c0-cde04dbc04b7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:29.721630 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-7016d8f2-4a47-463a-93c0-cde04dbc04b7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219" status: 200 len: 1362 microversion: 2.1 time: 0.198840 Sep 04 01:10:29.722208 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 438/897] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:29 2022] GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219 => generated 1362 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:30.501842 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-6d00aad2-89b9-4eda-b536-014d2fbed443 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:30.502460 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-6d00aad2-89b9-4eda-b536-014d2fbed443 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:30.509381 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6d00aad2-89b9-4eda-b536-014d2fbed443 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:30.509957 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6d00aad2-89b9-4eda-b536-014d2fbed443 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:30.510423 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6d00aad2-89b9-4eda-b536-014d2fbed443 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:30.511134 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6d00aad2-89b9-4eda-b536-014d2fbed443 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:30.511591 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6d00aad2-89b9-4eda-b536-014d2fbed443 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:30.512019 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6d00aad2-89b9-4eda-b536-014d2fbed443 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:30.576966 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-6d00aad2-89b9-4eda-b536-014d2fbed443 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:30.585281 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-6d00aad2-89b9-4eda-b536-014d2fbed443 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1622 microversion: 2.1 time: 0.087233 Sep 04 01:10:30.585873 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 460/898] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:30 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1622 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:30.748146 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-b2674976-e683-4348-8119-92d25ed8c34d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:30.748755 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-b2674976-e683-4348-8119-92d25ed8c34d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:30.757726 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b2674976-e683-4348-8119-92d25ed8c34d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:30.758208 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b2674976-e683-4348-8119-92d25ed8c34d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:30.759345 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b2674976-e683-4348-8119-92d25ed8c34d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:30.760063 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b2674976-e683-4348-8119-92d25ed8c34d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:30.760614 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b2674976-e683-4348-8119-92d25ed8c34d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:30.761038 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b2674976-e683-4348-8119-92d25ed8c34d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:30.811022 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-b17e09b4-33ed-4f91-b79f-abfde2d8ed56 req-db8c108a-5533-43c1-8d96-8326eeb05fcc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2570ee6c-f335-41bc-8e67-f89a62b28134", "name": "network-vif-deleted", "tag": "62f5079a-1d7d-4c5c-bdcd-0af337b6235b"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:10:30.823033 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-b17e09b4-33ed-4f91-b79f-abfde2d8ed56 req-db8c108a-5533-43c1-8d96-8326eeb05fcc service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:30.825231 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-b17e09b4-33ed-4f91-b79f-abfde2d8ed56 req-db8c108a-5533-43c1-8d96-8326eeb05fcc service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:30.825231 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-b17e09b4-33ed-4f91-b79f-abfde2d8ed56 req-db8c108a-5533-43c1-8d96-8326eeb05fcc service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:30.835402 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-b17e09b4-33ed-4f91-b79f-abfde2d8ed56 req-db8c108a-5533-43c1-8d96-8326eeb05fcc service nova] Creating event network-vif-deleted:62f5079a-1d7d-4c5c-bdcd-0af337b6235b for instance 2570ee6c-f335-41bc-8e67-f89a62b28134 on hv-cinder-maste Sep 04 01:10:30.840490 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-b17e09b4-33ed-4f91-b79f-abfde2d8ed56 req-db8c108a-5533-43c1-8d96-8326eeb05fcc service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034533 Sep 04 01:10:30.841110 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 461/899] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:10:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:10:30.916458 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-b2674976-e683-4348-8119-92d25ed8c34d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:30.918859 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:10:30.925834 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-b2674976-e683-4348-8119-92d25ed8c34d tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219" status: 200 len: 1362 microversion: 2.1 time: 0.182869 Sep 04 01:10:30.932837 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 439/900] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:30 2022] GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219 => generated 1362 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:31.611638 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-6c6292e3-c74c-4359-8e3d-9f6d8ac076fa tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:31.613290 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-6c6292e3-c74c-4359-8e3d-9f6d8ac076fa tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:31.621754 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6c6292e3-c74c-4359-8e3d-9f6d8ac076fa tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:31.622388 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6c6292e3-c74c-4359-8e3d-9f6d8ac076fa tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:31.622988 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6c6292e3-c74c-4359-8e3d-9f6d8ac076fa tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:31.623793 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6c6292e3-c74c-4359-8e3d-9f6d8ac076fa tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:31.624374 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6c6292e3-c74c-4359-8e3d-9f6d8ac076fa tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:31.624897 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6c6292e3-c74c-4359-8e3d-9f6d8ac076fa tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:31.706237 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-6c6292e3-c74c-4359-8e3d-9f6d8ac076fa tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:31.710166 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=100 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:10:31.715290 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-6c6292e3-c74c-4359-8e3d-9f6d8ac076fa tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1331 microversion: 2.1 time: 0.109479 Sep 04 01:10:31.715800 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 462/901] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:31 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1331 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:31.951675 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-81636a0f-94d9-48ad-9141-103d9a34c6d5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:31.952584 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-81636a0f-94d9-48ad-9141-103d9a34c6d5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:31.960993 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-81636a0f-94d9-48ad-9141-103d9a34c6d5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:31.961660 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-81636a0f-94d9-48ad-9141-103d9a34c6d5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:31.962925 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-81636a0f-94d9-48ad-9141-103d9a34c6d5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:31.963741 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-81636a0f-94d9-48ad-9141-103d9a34c6d5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:31.964294 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-81636a0f-94d9-48ad-9141-103d9a34c6d5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:31.971767 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-81636a0f-94d9-48ad-9141-103d9a34c6d5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:32.111250 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-81636a0f-94d9-48ad-9141-103d9a34c6d5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:32.121394 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-81636a0f-94d9-48ad-9141-103d9a34c6d5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219" status: 200 len: 1547 microversion: 2.1 time: 0.175223 Sep 04 01:10:32.125859 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 440/902] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:31 2022] GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219 => generated 1547 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:32.145194 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-f7c6a805-53ff-4e6b-b07d-8ae95744120f tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:32.148860 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-f7c6a805-53ff-4e6b-b07d-8ae95744120f tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:32.152623 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f7c6a805-53ff-4e6b-b07d-8ae95744120f tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:32.153124 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f7c6a805-53ff-4e6b-b07d-8ae95744120f tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:32.153577 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f7c6a805-53ff-4e6b-b07d-8ae95744120f tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:32.154248 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f7c6a805-53ff-4e6b-b07d-8ae95744120f tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:32.154693 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f7c6a805-53ff-4e6b-b07d-8ae95744120f tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:32.155109 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f7c6a805-53ff-4e6b-b07d-8ae95744120f tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:32.235717 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-f7c6a805-53ff-4e6b-b07d-8ae95744120f tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.094519 Sep 04 01:10:32.236391 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 463/903] 10.222.0.42 () {58 vars in 1275 bytes} [Sun Sep 4 01:10:32 2022] GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219/os-interface => generated 284 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:10:32.738981 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-b6064646-97b5-45e3-ab24-762cbab89d65 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:32.739584 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-b6064646-97b5-45e3-ab24-762cbab89d65 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:32.747553 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b6064646-97b5-45e3-ab24-762cbab89d65 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:32.747553 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b6064646-97b5-45e3-ab24-762cbab89d65 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:32.747805 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b6064646-97b5-45e3-ab24-762cbab89d65 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:32.748355 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b6064646-97b5-45e3-ab24-762cbab89d65 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:32.748802 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b6064646-97b5-45e3-ab24-762cbab89d65 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:32.749255 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b6064646-97b5-45e3-ab24-762cbab89d65 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:32.781011 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b6064646-97b5-45e3-ab24-762cbab89d65 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:32.781533 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b6064646-97b5-45e3-ab24-762cbab89d65 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:32.781960 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b6064646-97b5-45e3-ab24-762cbab89d65 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:32.782394 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.objects.instance [None req-b6064646-97b5-45e3-ab24-762cbab89d65 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lazy-loading 'fault' on Instance uuid 2570ee6c-f335-41bc-8e67-f89a62b28134 {{(pid=121037) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:10:32.824961 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-b6064646-97b5-45e3-ab24-762cbab89d65 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:32.832371 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-b6064646-97b5-45e3-ab24-762cbab89d65 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 200 len: 1336 microversion: 2.1 time: 0.097525 Sep 04 01:10:32.833507 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 441/904] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:32 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 1336 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:33.508676 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-c5b9b06a-d08d-4426-bde1-3da1d75b2534 req-8514feaf-769b-4b5b-a5a9-dd6c85566e4a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "52505c46-cb03-49ec-a7ba-696d7c6ac219", "tag": "73f5bdf6-a4ae-435e-9050-14b4d54218f9"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:10:33.518746 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-c5b9b06a-d08d-4426-bde1-3da1d75b2534 req-8514feaf-769b-4b5b-a5a9-dd6c85566e4a service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:33.519203 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-c5b9b06a-d08d-4426-bde1-3da1d75b2534 req-8514feaf-769b-4b5b-a5a9-dd6c85566e4a service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:33.519628 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-c5b9b06a-d08d-4426-bde1-3da1d75b2534 req-8514feaf-769b-4b5b-a5a9-dd6c85566e4a service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:33.530709 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-c5b9b06a-d08d-4426-bde1-3da1d75b2534 req-8514feaf-769b-4b5b-a5a9-dd6c85566e4a service nova] Creating event network-changed:73f5bdf6-a4ae-435e-9050-14b4d54218f9 for instance 52505c46-cb03-49ec-a7ba-696d7c6ac219 on hv-cinder-maste Sep 04 01:10:33.535573 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-c5b9b06a-d08d-4426-bde1-3da1d75b2534 req-8514feaf-769b-4b5b-a5a9-dd6c85566e4a service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.030421 Sep 04 01:10:33.536166 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 464/905] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:10:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:10:33.856999 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-b4f01950-aaaf-4e0b-a5b2-e2d409212194 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:33.857674 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-b4f01950-aaaf-4e0b-a5b2-e2d409212194 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2570ee6c-f335-41bc-8e67-f89a62b28134] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:33.864584 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b4f01950-aaaf-4e0b-a5b2-e2d409212194 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:33.865093 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b4f01950-aaaf-4e0b-a5b2-e2d409212194 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:33.865556 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b4f01950-aaaf-4e0b-a5b2-e2d409212194 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:33.866250 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b4f01950-aaaf-4e0b-a5b2-e2d409212194 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:33.866707 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b4f01950-aaaf-4e0b-a5b2-e2d409212194 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:33.867132 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b4f01950-aaaf-4e0b-a5b2-e2d409212194 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:33.882236 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.wsgi [None req-b4f01950-aaaf-4e0b-a5b2-e2d409212194 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] HTTP exception thrown: Instance 2570ee6c-f335-41bc-8e67-f89a62b28134 could not be found. Sep 04 01:10:33.884186 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-b4f01950-aaaf-4e0b-a5b2-e2d409212194 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Request method failure captured: Sep 04 01:10:33.884186 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: request: GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 HTTP/1.1 Sep 04 01:10:33.884186 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept: application/json Sep 04 01:10:33.884186 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept-Encoding: identity Sep 04 01:10:33.884186 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Connection: close Sep 04 01:10:33.884186 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Length: 0 Sep 04 01:10:33.884186 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Type: application/json Sep 04 01:10:33.884186 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Host: 10.222.0.42 Sep 04 01:10:33.884186 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Openstack-System-Scope: None Sep 04 01:10:33.884186 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: User-Agent: python-urllib3/1.26.12 Sep 04 01:10:33.884186 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Auth-Token: gAAAAABjE_nMCq4zD8BmB6mF8h26w0y_IFei-x7tMbI5vcKfWFaeAi75nOGGI72_ZHVfHvHOCK5qrlGll_NneJYvZkWgGCkELi_QIwkIIEFfz91_ouiYPU3bkvjmjKoIsPt_9g8qfiCyLaERnndIFBkeIUe0g0NvmODNU753JT7NKZpvzaTLcsI Sep 04 01:10:33.884186 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Id: None Sep 04 01:10:33.884186 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Name: None Sep 04 01:10:33.884186 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Identity-Status: Confirmed Sep 04 01:10:33.884186 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Is-Admin-Project: True Sep 04 01:10:33.884186 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Id: default Sep 04 01:10:33.884186 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Name: Default Sep 04 01:10:33.884186 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Id: db72b63281384bd883b04d05bcc35677 Sep 04 01:10:33.884186 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1367456352 Sep 04 01:10:33.884186 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Role: reader,member Sep 04 01:10:33.884186 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Roles: reader,member Sep 04 01:10:33.885804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/db72b63281384bd883b04d05bcc35677"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_db72b63281384bd883b04d05bcc35677", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/db72b63281384bd883b04d05bcc35677"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/db72b63281384bd883b04d05bcc35677"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:10:33.885804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1367456352 Sep 04 01:10:33.885804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Id: db72b63281384bd883b04d05bcc35677 Sep 04 01:10:33.885804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1367456352 Sep 04 01:10:33.885804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User: tempest-ServerRescueNegativeTestJSON-1367456352-project Sep 04 01:10:33.885804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Id: default Sep 04 01:10:33.885804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Name: Default Sep 04 01:10:33.885804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Id: bd13df3a2d0e4182bbd86dab7035c371 Sep 04 01:10:33.885804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1367456352-project Sep 04 01:10:33.885804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: method: > Sep 04 01:10:33.885804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: exception: Instance 2570ee6c-f335-41bc-8e67-f89a62b28134 could not be found. Sep 04 01:10:33.885804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: action_args: {'id': '2570ee6c-f335-41bc-8e67-f89a62b28134'} Sep 04 01:10:33.886563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:10:33.886563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:10:33.886563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:10:33.886563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:10:33.886563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:10:33.886563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:10:33.886563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:10:33.886563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:10:33.886563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:10:33.886563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:10:33.886563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2570ee6c-f335-41bc-8e67-f89a62b28134 could not be found. Sep 04 01:10:33.886563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:10:33.886563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:10:33.886563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:10:33.886563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:10:33.886563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:10:33.886563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:10:33.886563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:10:33.886563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:10:33.886563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:10:33.887757 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:10:33.887757 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 04 01:10:33.887757 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 04 01:10:33.887757 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:10:33.887757 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:10:33.887757 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:10:33.887757 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:10:33.887757 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2570ee6c-f335-41bc-8e67-f89a62b28134 could not be found. Sep 04 01:10:33.887757 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:10:33.887757 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:10:33.887757 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:10:33.887757 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:10:33.887757 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:10:33.887757 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:10:33.887757 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:10:33.887757 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:10:33.887757 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2570ee6c-f335-41bc-8e67-f89a62b28134 could not be found. Sep 04 01:10:33.887757 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:10:33.888874 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-b4f01950-aaaf-4e0b-a5b2-e2d409212194 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Returning 404 to user: Instance 2570ee6c-f335-41bc-8e67-f89a62b28134 could not be found. {{(pid=121037) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:10:33.888874 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-b4f01950-aaaf-4e0b-a5b2-e2d409212194 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134" status: 404 len: 111 microversion: 2.1 time: 0.033820 Sep 04 01:10:33.889972 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 442/906] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:33 2022] GET /compute/v2.1/servers/2570ee6c-f335-41bc-8e67-f89a62b28134 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 04 01:10:33.896695 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-2d8b9687-2292-4eee-800a-21bc7891c6a3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '.version_select at 0x7fc10c5698b0>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:33.953449 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-2d8b9687-2292-4eee-800a-21bc7891c6a3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1002954839" status: 202 len: 0 microversion: 2.1 time: 0.060779 Sep 04 01:10:33.954763 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 465/907] 10.222.0.42 () {58 vars in 1221 bytes} [Sun Sep 4 01:10:33 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1002954839 => generated 0 bytes in 64 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 04 01:10:37.298458 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-c6a4a64b-9f66-44ce-bc62-0ff1d4f2625f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:37.298956 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-c6a4a64b-9f66-44ce-bc62-0ff1d4f2625f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:37.305683 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c6a4a64b-9f66-44ce-bc62-0ff1d4f2625f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:37.306153 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c6a4a64b-9f66-44ce-bc62-0ff1d4f2625f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:37.306582 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c6a4a64b-9f66-44ce-bc62-0ff1d4f2625f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:37.307259 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c6a4a64b-9f66-44ce-bc62-0ff1d4f2625f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:37.307732 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c6a4a64b-9f66-44ce-bc62-0ff1d4f2625f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:37.308910 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c6a4a64b-9f66-44ce-bc62-0ff1d4f2625f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:37.333285 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-c6a4a64b-9f66-44ce-bc62-0ff1d4f2625f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Going to try to terminate instance {{(pid=121037) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 04 01:10:37.592265 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-c6a4a64b-9f66-44ce-bc62-0ff1d4f2625f tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "DELETE /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 204 len: 0 microversion: 2.1 time: 0.297336 Sep 04 01:10:37.592862 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 443/908] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 4 01:10:37 2022] DELETE /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 04 01:10:37.601150 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-fc0ddd81-1333-4626-9686-ac369a10f979 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:37.601892 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-fc0ddd81-1333-4626-9686-ac369a10f979 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:37.611041 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fc0ddd81-1333-4626-9686-ac369a10f979 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:37.611561 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fc0ddd81-1333-4626-9686-ac369a10f979 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:37.612004 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fc0ddd81-1333-4626-9686-ac369a10f979 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:37.612796 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fc0ddd81-1333-4626-9686-ac369a10f979 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:37.613552 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fc0ddd81-1333-4626-9686-ac369a10f979 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:37.613986 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fc0ddd81-1333-4626-9686-ac369a10f979 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:37.745892 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-fc0ddd81-1333-4626-9686-ac369a10f979 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:37.753739 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-fc0ddd81-1333-4626-9686-ac369a10f979 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1505 microversion: 2.1 time: 0.156599 Sep 04 01:10:37.754312 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 466/909] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:37 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1505 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:38.777511 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-6a0d03b1-23b9-43f3-9063-f0e17b4b2102 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:38.778132 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-6a0d03b1-23b9-43f3-9063-f0e17b4b2102 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:38.784520 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6a0d03b1-23b9-43f3-9063-f0e17b4b2102 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:38.785002 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6a0d03b1-23b9-43f3-9063-f0e17b4b2102 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:38.785468 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6a0d03b1-23b9-43f3-9063-f0e17b4b2102 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:38.786160 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6a0d03b1-23b9-43f3-9063-f0e17b4b2102 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:38.786615 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6a0d03b1-23b9-43f3-9063-f0e17b4b2102 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:38.787041 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6a0d03b1-23b9-43f3-9063-f0e17b4b2102 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:38.922954 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-6a0d03b1-23b9-43f3-9063-f0e17b4b2102 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:38.932155 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-6a0d03b1-23b9-43f3-9063-f0e17b4b2102 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1505 microversion: 2.1 time: 0.158927 Sep 04 01:10:38.934706 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 444/910] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:38 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1505 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:39.962779 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-37f9a57e-dd5e-4363-8c58-d766def6e4c3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:39.963383 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-37f9a57e-dd5e-4363-8c58-d766def6e4c3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:39.970035 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-37f9a57e-dd5e-4363-8c58-d766def6e4c3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:39.970507 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-37f9a57e-dd5e-4363-8c58-d766def6e4c3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:39.970938 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-37f9a57e-dd5e-4363-8c58-d766def6e4c3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:39.971617 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-37f9a57e-dd5e-4363-8c58-d766def6e4c3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:39.972072 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-37f9a57e-dd5e-4363-8c58-d766def6e4c3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:39.972489 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-37f9a57e-dd5e-4363-8c58-d766def6e4c3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:40.100553 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-37f9a57e-dd5e-4363-8c58-d766def6e4c3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:40.109396 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-37f9a57e-dd5e-4363-8c58-d766def6e4c3 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1505 microversion: 2.1 time: 0.150423 Sep 04 01:10:40.109978 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 467/911] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:39 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1505 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:40.834168 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-a3cdf759-9127-459c-b33c-a4cbec337df0 req-ec949e94-c629-4807-8a36-9be2a5855a5c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "62c4a968-ecfa-4842-8cab-8aaeb7ca7a64", "name": "network-vif-deleted", "tag": "addc777a-4bdc-4153-92fa-8184c15d2cd1"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:10:40.844650 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-a3cdf759-9127-459c-b33c-a4cbec337df0 req-ec949e94-c629-4807-8a36-9be2a5855a5c service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:40.845178 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-a3cdf759-9127-459c-b33c-a4cbec337df0 req-ec949e94-c629-4807-8a36-9be2a5855a5c service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:40.845608 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-a3cdf759-9127-459c-b33c-a4cbec337df0 req-ec949e94-c629-4807-8a36-9be2a5855a5c service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:40.857758 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-a3cdf759-9127-459c-b33c-a4cbec337df0 req-ec949e94-c629-4807-8a36-9be2a5855a5c service nova] Creating event network-vif-deleted:addc777a-4bdc-4153-92fa-8184c15d2cd1 for instance 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 on hv-cinder-maste Sep 04 01:10:40.862947 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-a3cdf759-9127-459c-b33c-a4cbec337df0 req-ec949e94-c629-4807-8a36-9be2a5855a5c service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.093373 Sep 04 01:10:40.864077 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 445/912] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:10:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:10:41.133580 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-f9821ed1-0f35-418b-8223-3898dc910beb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:41.134297 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-f9821ed1-0f35-418b-8223-3898dc910beb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:41.141717 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f9821ed1-0f35-418b-8223-3898dc910beb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:41.144358 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f9821ed1-0f35-418b-8223-3898dc910beb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:41.144358 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f9821ed1-0f35-418b-8223-3898dc910beb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:41.144358 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f9821ed1-0f35-418b-8223-3898dc910beb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:41.144358 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f9821ed1-0f35-418b-8223-3898dc910beb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:41.144714 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f9821ed1-0f35-418b-8223-3898dc910beb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:41.178594 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f9821ed1-0f35-418b-8223-3898dc910beb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:41.179074 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f9821ed1-0f35-418b-8223-3898dc910beb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:41.179515 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f9821ed1-0f35-418b-8223-3898dc910beb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:41.179962 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.objects.instance [None req-f9821ed1-0f35-418b-8223-3898dc910beb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lazy-loading 'fault' on Instance uuid 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 {{(pid=121036) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:10:41.227215 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-f9821ed1-0f35-418b-8223-3898dc910beb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:41.235115 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-f9821ed1-0f35-418b-8223-3898dc910beb tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 200 len: 1310 microversion: 2.1 time: 0.105433 Sep 04 01:10:41.235688 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 468/913] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:41 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 1310 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:42.259961 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-454e6667-990d-4511-a585-6f464d043574 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:42.260651 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-454e6667-990d-4511-a585-6f464d043574 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:42.268627 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-454e6667-990d-4511-a585-6f464d043574 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:42.269132 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-454e6667-990d-4511-a585-6f464d043574 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:42.269578 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-454e6667-990d-4511-a585-6f464d043574 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:42.270520 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-454e6667-990d-4511-a585-6f464d043574 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:42.270726 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-454e6667-990d-4511-a585-6f464d043574 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:42.271156 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-454e6667-990d-4511-a585-6f464d043574 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:42.284422 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.wsgi [None req-454e6667-990d-4511-a585-6f464d043574 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] HTTP exception thrown: Instance 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 could not be found. Sep 04 01:10:42.286062 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-454e6667-990d-4511-a585-6f464d043574 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Request method failure captured: Sep 04 01:10:42.286062 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: request: GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 HTTP/1.1 Sep 04 01:10:42.286062 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept: application/json Sep 04 01:10:42.286062 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept-Encoding: identity Sep 04 01:10:42.286062 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Connection: close Sep 04 01:10:42.286062 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Length: 0 Sep 04 01:10:42.286062 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Type: application/json Sep 04 01:10:42.286062 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Host: 10.222.0.42 Sep 04 01:10:42.286062 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Openstack-System-Scope: None Sep 04 01:10:42.286062 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: User-Agent: python-urllib3/1.26.12 Sep 04 01:10:42.286062 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Auth-Token: gAAAAABjE_nMCq4zD8BmB6mF8h26w0y_IFei-x7tMbI5vcKfWFaeAi75nOGGI72_ZHVfHvHOCK5qrlGll_NneJYvZkWgGCkELi_QIwkIIEFfz91_ouiYPU3bkvjmjKoIsPt_9g8qfiCyLaERnndIFBkeIUe0g0NvmODNU753JT7NKZpvzaTLcsI Sep 04 01:10:42.286062 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Id: None Sep 04 01:10:42.286062 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Name: None Sep 04 01:10:42.286062 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Identity-Status: Confirmed Sep 04 01:10:42.286062 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Is-Admin-Project: True Sep 04 01:10:42.286062 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Id: default Sep 04 01:10:42.286062 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Name: Default Sep 04 01:10:42.286062 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Id: db72b63281384bd883b04d05bcc35677 Sep 04 01:10:42.286062 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1367456352 Sep 04 01:10:42.286062 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Role: reader,member Sep 04 01:10:42.286062 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Roles: reader,member Sep 04 01:10:42.287794 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/db72b63281384bd883b04d05bcc35677"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_db72b63281384bd883b04d05bcc35677", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/db72b63281384bd883b04d05bcc35677"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/db72b63281384bd883b04d05bcc35677"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:10:42.287794 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1367456352 Sep 04 01:10:42.287794 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Id: db72b63281384bd883b04d05bcc35677 Sep 04 01:10:42.287794 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1367456352 Sep 04 01:10:42.287794 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User: tempest-ServerRescueNegativeTestJSON-1367456352-project Sep 04 01:10:42.287794 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Id: default Sep 04 01:10:42.287794 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Name: Default Sep 04 01:10:42.287794 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Id: bd13df3a2d0e4182bbd86dab7035c371 Sep 04 01:10:42.287794 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1367456352-project Sep 04 01:10:42.287794 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: method: > Sep 04 01:10:42.287794 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: exception: Instance 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 could not be found. Sep 04 01:10:42.287794 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: action_args: {'id': '62c4a968-ecfa-4842-8cab-8aaeb7ca7a64'} Sep 04 01:10:42.288733 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:10:42.288733 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:10:42.288733 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:10:42.288733 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:10:42.288733 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:10:42.288733 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:10:42.288733 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:10:42.288733 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:10:42.288733 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:10:42.288733 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:10:42.288733 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 could not be found. Sep 04 01:10:42.288733 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:10:42.288733 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:10:42.288733 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:10:42.288733 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:10:42.288733 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:10:42.288733 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:10:42.288733 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:10:42.288733 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:10:42.288733 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:10:42.290307 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:10:42.290307 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 04 01:10:42.290307 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 04 01:10:42.290307 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:10:42.290307 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:10:42.290307 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:10:42.290307 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:10:42.290307 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 could not be found. Sep 04 01:10:42.290307 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:10:42.290307 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:10:42.290307 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:10:42.290307 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:10:42.290307 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:10:42.290307 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:10:42.290307 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:10:42.290307 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:10:42.290307 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 could not be found. Sep 04 01:10:42.290307 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:10:42.291783 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-454e6667-990d-4511-a585-6f464d043574 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Returning 404 to user: Instance 62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 could not be found. {{(pid=121037) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:10:42.291783 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-454e6667-990d-4511-a585-6f464d043574 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64" status: 404 len: 111 microversion: 2.1 time: 0.032261 Sep 04 01:10:42.291783 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 446/914] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:42 2022] GET /compute/v2.1/servers/62c4a968-ecfa-4842-8cab-8aaeb7ca7a64 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 04 01:10:42.296202 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-a7a7c69e-1e72-4c2d-a7f3-c42d048f4137 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:42.296696 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-a7a7c69e-1e72-4c2d-a7f3-c42d048f4137 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:42.302826 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7a7c69e-1e72-4c2d-a7f3-c42d048f4137 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:42.303300 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7a7c69e-1e72-4c2d-a7f3-c42d048f4137 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:42.303730 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7a7c69e-1e72-4c2d-a7f3-c42d048f4137 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:42.304409 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7a7c69e-1e72-4c2d-a7f3-c42d048f4137 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:42.304854 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7a7c69e-1e72-4c2d-a7f3-c42d048f4137 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:42.305302 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7a7c69e-1e72-4c2d-a7f3-c42d048f4137 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:42.330424 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-a7a7c69e-1e72-4c2d-a7f3-c42d048f4137 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Going to try to terminate instance {{(pid=121036) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 04 01:10:42.558502 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-a7a7c69e-1e72-4c2d-a7f3-c42d048f4137 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "DELETE /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 204 len: 0 microversion: 2.1 time: 0.265342 Sep 04 01:10:42.559126 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 469/915] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 4 01:10:42 2022] DELETE /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 04 01:10:42.567915 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-214a889b-4ca0-410d-802a-7f9be2497f68 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:42.568525 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-214a889b-4ca0-410d-802a-7f9be2497f68 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:42.575703 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-214a889b-4ca0-410d-802a-7f9be2497f68 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:42.578387 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-214a889b-4ca0-410d-802a-7f9be2497f68 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:42.578387 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-214a889b-4ca0-410d-802a-7f9be2497f68 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:42.578387 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-214a889b-4ca0-410d-802a-7f9be2497f68 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:42.578387 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-214a889b-4ca0-410d-802a-7f9be2497f68 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:42.578782 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-214a889b-4ca0-410d-802a-7f9be2497f68 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:42.710765 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-214a889b-4ca0-410d-802a-7f9be2497f68 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:42.719182 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-214a889b-4ca0-410d-802a-7f9be2497f68 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1522 microversion: 2.1 time: 0.155331 Sep 04 01:10:42.719748 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 447/916] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:42 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1522 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:43.742526 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-e281327b-d5cc-4165-a176-cd9b1111fc57 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:43.743223 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-e281327b-d5cc-4165-a176-cd9b1111fc57 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:43.745807 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_api:SELECT=78,nova_api:INSERT=3,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:10:43.750487 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e281327b-d5cc-4165-a176-cd9b1111fc57 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:43.750964 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e281327b-d5cc-4165-a176-cd9b1111fc57 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:43.751406 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e281327b-d5cc-4165-a176-cd9b1111fc57 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:43.752092 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e281327b-d5cc-4165-a176-cd9b1111fc57 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:43.752547 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e281327b-d5cc-4165-a176-cd9b1111fc57 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:43.752977 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e281327b-d5cc-4165-a176-cd9b1111fc57 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:43.889352 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e281327b-d5cc-4165-a176-cd9b1111fc57 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:43.896875 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-e281327b-d5cc-4165-a176-cd9b1111fc57 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1522 microversion: 2.1 time: 0.158912 Sep 04 01:10:43.897466 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 470/917] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:43 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1522 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:44.920365 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-ffbd4cff-7cb6-46d1-9d2e-48f87d00acae tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:44.920981 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-ffbd4cff-7cb6-46d1-9d2e-48f87d00acae tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:44.927624 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ffbd4cff-7cb6-46d1-9d2e-48f87d00acae tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:44.928141 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ffbd4cff-7cb6-46d1-9d2e-48f87d00acae tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:44.928577 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ffbd4cff-7cb6-46d1-9d2e-48f87d00acae tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:44.929312 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ffbd4cff-7cb6-46d1-9d2e-48f87d00acae tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:44.929747 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ffbd4cff-7cb6-46d1-9d2e-48f87d00acae tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:44.930174 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ffbd4cff-7cb6-46d1-9d2e-48f87d00acae tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:45.056317 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-ffbd4cff-7cb6-46d1-9d2e-48f87d00acae tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:45.064085 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-ffbd4cff-7cb6-46d1-9d2e-48f87d00acae tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1522 microversion: 2.1 time: 0.148121 Sep 04 01:10:45.064647 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 448/918] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:44 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1522 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:46.087062 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-2c0a3b87-ab70-49a1-8235-6dfc5e5a26d0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:46.087684 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-2c0a3b87-ab70-49a1-8235-6dfc5e5a26d0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:46.094908 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2c0a3b87-ab70-49a1-8235-6dfc5e5a26d0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:46.104072 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2c0a3b87-ab70-49a1-8235-6dfc5e5a26d0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:46.104072 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2c0a3b87-ab70-49a1-8235-6dfc5e5a26d0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:46.104072 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2c0a3b87-ab70-49a1-8235-6dfc5e5a26d0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:46.104072 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2c0a3b87-ab70-49a1-8235-6dfc5e5a26d0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:46.104484 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2c0a3b87-ab70-49a1-8235-6dfc5e5a26d0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:46.226127 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-2c0a3b87-ab70-49a1-8235-6dfc5e5a26d0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:46.234705 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-2c0a3b87-ab70-49a1-8235-6dfc5e5a26d0 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1522 microversion: 2.1 time: 0.151487 Sep 04 01:10:46.241635 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 471/919] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:46 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1522 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:46.830809 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-8dbb935e-5707-4636-94dc-d7faf61afb43 req-7aab7de8-808e-4e37-b234-7ae6fe857f4b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2f7a7e72-1f16-4ab4-b826-d376d39be4c2", "name": "network-vif-deleted", "tag": "28efc83e-9838-45a7-9fd3-3625eeff4b09"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:10:46.836231 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_api:SELECT=66,nova_api:UPDATE=2 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:10:46.841871 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-8dbb935e-5707-4636-94dc-d7faf61afb43 req-7aab7de8-808e-4e37-b234-7ae6fe857f4b service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:46.842344 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-8dbb935e-5707-4636-94dc-d7faf61afb43 req-7aab7de8-808e-4e37-b234-7ae6fe857f4b service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:46.842851 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-8dbb935e-5707-4636-94dc-d7faf61afb43 req-7aab7de8-808e-4e37-b234-7ae6fe857f4b service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:46.853595 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-8dbb935e-5707-4636-94dc-d7faf61afb43 req-7aab7de8-808e-4e37-b234-7ae6fe857f4b service nova] Creating event network-vif-deleted:28efc83e-9838-45a7-9fd3-3625eeff4b09 for instance 2f7a7e72-1f16-4ab4-b826-d376d39be4c2 on hv-cinder-maste Sep 04 01:10:46.859154 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-8dbb935e-5707-4636-94dc-d7faf61afb43 req-7aab7de8-808e-4e37-b234-7ae6fe857f4b service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031943 Sep 04 01:10:46.859745 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 449/920] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:10:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:10:47.258907 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-10840e86-4eb6-4311-9338-44881706b715 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:47.259562 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-10840e86-4eb6-4311-9338-44881706b715 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:47.266481 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-10840e86-4eb6-4311-9338-44881706b715 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:47.266964 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-10840e86-4eb6-4311-9338-44881706b715 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:47.267402 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-10840e86-4eb6-4311-9338-44881706b715 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:47.268090 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-10840e86-4eb6-4311-9338-44881706b715 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:47.268543 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-10840e86-4eb6-4311-9338-44881706b715 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:47.268969 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-10840e86-4eb6-4311-9338-44881706b715 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:47.302236 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-10840e86-4eb6-4311-9338-44881706b715 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:47.302721 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-10840e86-4eb6-4311-9338-44881706b715 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:47.303294 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-10840e86-4eb6-4311-9338-44881706b715 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:47.303746 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.objects.instance [None req-10840e86-4eb6-4311-9338-44881706b715 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lazy-loading 'fault' on Instance uuid 2f7a7e72-1f16-4ab4-b826-d376d39be4c2 {{(pid=121036) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:10:47.347854 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-10840e86-4eb6-4311-9338-44881706b715 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd13df3a2d0e4182bbd86dab7035c371', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'db72b63281384bd883b04d05bcc35677', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:10:47.360558 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-10840e86-4eb6-4311-9338-44881706b715 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 200 len: 1312 microversion: 2.1 time: 0.102331 Sep 04 01:10:47.360558 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 472/921] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:47 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 1312 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:10:48.302608 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-2c4a5a6b-910f-49df-aec7-fee2e0050cea tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:48.303172 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-2c4a5a6b-910f-49df-aec7-fee2e0050cea tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 16b0498f-4cb8-491a-92cf-dac9890c7b4a] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:48.309944 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2c4a5a6b-910f-49df-aec7-fee2e0050cea tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:48.310414 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2c4a5a6b-910f-49df-aec7-fee2e0050cea tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:48.310847 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2c4a5a6b-910f-49df-aec7-fee2e0050cea tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:48.311519 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2c4a5a6b-910f-49df-aec7-fee2e0050cea tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:48.311962 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2c4a5a6b-910f-49df-aec7-fee2e0050cea tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:48.312377 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2c4a5a6b-910f-49df-aec7-fee2e0050cea tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:48.335450 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-2c4a5a6b-910f-49df-aec7-fee2e0050cea tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a/ips" status: 200 len: 150 microversion: 2.1 time: 0.136699 Sep 04 01:10:48.335995 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 450/922] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Sep 4 01:10:48 2022] GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a/ips => generated 150 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:10:48.383045 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-77cab4cd-0f50-41cd-af0f-19cad4214507 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:48.383685 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-77cab4cd-0f50-41cd-af0f-19cad4214507 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] [instance: 2f7a7e72-1f16-4ab4-b826-d376d39be4c2] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:48.390556 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-77cab4cd-0f50-41cd-af0f-19cad4214507 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:48.391045 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-77cab4cd-0f50-41cd-af0f-19cad4214507 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:48.391491 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-77cab4cd-0f50-41cd-af0f-19cad4214507 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:48.392188 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-77cab4cd-0f50-41cd-af0f-19cad4214507 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:48.392641 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-77cab4cd-0f50-41cd-af0f-19cad4214507 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:48.393095 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-77cab4cd-0f50-41cd-af0f-19cad4214507 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:48.406388 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.wsgi [None req-77cab4cd-0f50-41cd-af0f-19cad4214507 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] HTTP exception thrown: Instance 2f7a7e72-1f16-4ab4-b826-d376d39be4c2 could not be found. Sep 04 01:10:48.408275 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-77cab4cd-0f50-41cd-af0f-19cad4214507 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Request method failure captured: Sep 04 01:10:48.408275 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: request: GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 HTTP/1.1 Sep 04 01:10:48.408275 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept: application/json Sep 04 01:10:48.408275 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept-Encoding: identity Sep 04 01:10:48.408275 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Connection: close Sep 04 01:10:48.408275 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Length: 0 Sep 04 01:10:48.408275 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Type: application/json Sep 04 01:10:48.408275 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Host: 10.222.0.42 Sep 04 01:10:48.408275 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Openstack-System-Scope: None Sep 04 01:10:48.408275 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: User-Agent: python-urllib3/1.26.12 Sep 04 01:10:48.408275 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Auth-Token: gAAAAABjE_nMCq4zD8BmB6mF8h26w0y_IFei-x7tMbI5vcKfWFaeAi75nOGGI72_ZHVfHvHOCK5qrlGll_NneJYvZkWgGCkELi_QIwkIIEFfz91_ouiYPU3bkvjmjKoIsPt_9g8qfiCyLaERnndIFBkeIUe0g0NvmODNU753JT7NKZpvzaTLcsI Sep 04 01:10:48.408275 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Id: None Sep 04 01:10:48.408275 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Name: None Sep 04 01:10:48.408275 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Identity-Status: Confirmed Sep 04 01:10:48.408275 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Is-Admin-Project: True Sep 04 01:10:48.408275 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Id: default Sep 04 01:10:48.408275 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Name: Default Sep 04 01:10:48.408275 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Id: db72b63281384bd883b04d05bcc35677 Sep 04 01:10:48.408275 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1367456352 Sep 04 01:10:48.408275 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Role: reader,member Sep 04 01:10:48.408275 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Roles: reader,member Sep 04 01:10:48.409622 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/db72b63281384bd883b04d05bcc35677"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_db72b63281384bd883b04d05bcc35677", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/db72b63281384bd883b04d05bcc35677"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/db72b63281384bd883b04d05bcc35677"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:10:48.409622 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1367456352 Sep 04 01:10:48.409622 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Id: db72b63281384bd883b04d05bcc35677 Sep 04 01:10:48.409622 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1367456352 Sep 04 01:10:48.409622 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User: tempest-ServerRescueNegativeTestJSON-1367456352-project Sep 04 01:10:48.409622 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Id: default Sep 04 01:10:48.409622 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Name: Default Sep 04 01:10:48.409622 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Id: bd13df3a2d0e4182bbd86dab7035c371 Sep 04 01:10:48.409622 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1367456352-project Sep 04 01:10:48.409622 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: method: > Sep 04 01:10:48.409622 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: exception: Instance 2f7a7e72-1f16-4ab4-b826-d376d39be4c2 could not be found. Sep 04 01:10:48.409622 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: action_args: {'id': '2f7a7e72-1f16-4ab4-b826-d376d39be4c2'} Sep 04 01:10:48.414128 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:10:48.414128 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:10:48.414128 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:10:48.414128 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:10:48.414128 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:10:48.414128 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:10:48.414128 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:10:48.414128 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:10:48.414128 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:10:48.414128 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:10:48.414128 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2f7a7e72-1f16-4ab4-b826-d376d39be4c2 could not be found. Sep 04 01:10:48.414128 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:10:48.414128 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:10:48.414128 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:10:48.414128 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:10:48.414128 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:10:48.414128 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:10:48.414128 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:10:48.414128 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:10:48.414128 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:10:48.417700 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:10:48.417700 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 04 01:10:48.417700 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 04 01:10:48.417700 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:10:48.417700 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:10:48.417700 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:10:48.417700 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:10:48.417700 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2f7a7e72-1f16-4ab4-b826-d376d39be4c2 could not be found. Sep 04 01:10:48.417700 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:10:48.417700 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:10:48.417700 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:10:48.417700 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:10:48.417700 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:10:48.417700 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:10:48.417700 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:10:48.417700 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:10:48.417700 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2f7a7e72-1f16-4ab4-b826-d376d39be4c2 could not be found. Sep 04 01:10:48.417700 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:10:48.420201 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-77cab4cd-0f50-41cd-af0f-19cad4214507 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] Returning 404 to user: Instance 2f7a7e72-1f16-4ab4-b826-d376d39be4c2 could not be found. {{(pid=121036) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:10:48.420201 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-77cab4cd-0f50-41cd-af0f-19cad4214507 tempest-ServerRescueNegativeTestJSON-1367456352 tempest-ServerRescueNegativeTestJSON-1367456352-project] 10.222.0.42 "GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2" status: 404 len: 111 microversion: 2.1 time: 0.030419 Sep 04 01:10:48.420201 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 473/923] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:10:48 2022] GET /compute/v2.1/servers/2f7a7e72-1f16-4ab4-b826-d376d39be4c2 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 04 01:10:49.914187 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-0563b7dd-e244-4df2-b9ce-859012a98fb0 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "2b938702-b38c-4b5f-9d35-2a9c98f3b495"}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:10:49.916802 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-0563b7dd-e244-4df2-b9ce-859012a98fb0 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 16b0498f-4cb8-491a-92cf-dac9890c7b4a] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:49.924013 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0563b7dd-e244-4df2-b9ce-859012a98fb0 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:49.924561 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0563b7dd-e244-4df2-b9ce-859012a98fb0 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:49.925053 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0563b7dd-e244-4df2-b9ce-859012a98fb0 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:49.925853 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0563b7dd-e244-4df2-b9ce-859012a98fb0 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:49.926370 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0563b7dd-e244-4df2-b9ce-859012a98fb0 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:49.926852 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0563b7dd-e244-4df2-b9ce-859012a98fb0 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:50.671176 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-0563b7dd-e244-4df2-b9ce-859012a98fb0 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "POST /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.760757 Sep 04 01:10:50.671795 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 451/924] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Sep 4 01:10:49 2022] POST /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a/os-volume_attachments => generated 194 bytes in 762 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:10:58.244275 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-dd5b46b8-7b11-4378-8cdb-bc49be89193c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "2b938702-b38c-4b5f-9d35-2a9c98f3b495"}} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:10:58.247603 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-dd5b46b8-7b11-4378-8cdb-bc49be89193c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 16b0498f-4cb8-491a-92cf-dac9890c7b4a] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:58.254486 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dd5b46b8-7b11-4378-8cdb-bc49be89193c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:58.254965 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dd5b46b8-7b11-4378-8cdb-bc49be89193c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:58.255405 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dd5b46b8-7b11-4378-8cdb-bc49be89193c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:58.256093 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dd5b46b8-7b11-4378-8cdb-bc49be89193c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:58.256553 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dd5b46b8-7b11-4378-8cdb-bc49be89193c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:58.256978 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dd5b46b8-7b11-4378-8cdb-bc49be89193c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:58.338898 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.wsgi [None req-dd5b46b8-7b11-4378-8cdb-bc49be89193c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] HTTP exception thrown: Invalid volume: volume 2b938702-b38c-4b5f-9d35-2a9c98f3b495 is already attached to instances: 16b0498f-4cb8-491a-92cf-dac9890c7b4a Sep 04 01:10:58.340564 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-dd5b46b8-7b11-4378-8cdb-bc49be89193c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Request method failure captured: Sep 04 01:10:58.340564 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: request: POST /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a/os-volume_attachments HTTP/1.1 Sep 04 01:10:58.340564 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept: application/json Sep 04 01:10:58.340564 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept-Encoding: identity Sep 04 01:10:58.340564 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Connection: close Sep 04 01:10:58.340564 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Length: 74 Sep 04 01:10:58.340564 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Type: application/json Sep 04 01:10:58.340564 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Host: 10.222.0.42 Sep 04 01:10:58.340564 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Openstack-System-Scope: None Sep 04 01:10:58.340564 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: User-Agent: python-urllib3/1.26.12 Sep 04 01:10:58.340564 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Auth-Token: gAAAAABjE_nLdulKsCh7rP69EyY8StqBiRnQNFlCX5VoWZ9XzP_YTEis0NNuYGIELGjVYA3FFaOCC9PLYuhPddV3AgDhoAQw_S2o9MK0q1A57ehVT3gYS6mSkklOOLScx3a8r0TUFxeOMhMymWFJt03FSLp4K4nMUe8FttsVqGRsky_ik39T3Zg Sep 04 01:10:58.340564 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Id: None Sep 04 01:10:58.340564 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Name: None Sep 04 01:10:58.340564 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Identity-Status: Confirmed Sep 04 01:10:58.340564 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Is-Admin-Project: True Sep 04 01:10:58.340564 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Id: default Sep 04 01:10:58.340564 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Name: Default Sep 04 01:10:58.340564 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Id: 287d111f08ac4b05bd5adec9c40d0620 Sep 04 01:10:58.340564 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Name: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:10:58.340564 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Role: reader,member Sep 04 01:10:58.340564 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Roles: reader,member Sep 04 01:10:58.342107 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_287d111f08ac4b05bd5adec9c40d0620", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:10:58.342107 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:10:58.342107 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Id: 287d111f08ac4b05bd5adec9c40d0620 Sep 04 01:10:58.342107 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:10:58.342107 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User: tempest-AttachVolumeNegativeTest-825474236-project Sep 04 01:10:58.342107 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Id: default Sep 04 01:10:58.342107 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Name: Default Sep 04 01:10:58.342107 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Id: f5e4723258c64ce5885154dded2d3d43 Sep 04 01:10:58.342107 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Name: tempest-AttachVolumeNegativeTest-825474236-project Sep 04 01:10:58.342107 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Sep 04 01:10:58.342107 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: {"volumeAttachment": {"volumeId": "2b938702-b38c-4b5f-9d35-2a9c98f3b495"}} Sep 04 01:10:58.342107 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: method: > Sep 04 01:10:58.342925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: exception: Invalid volume: volume 2b938702-b38c-4b5f-9d35-2a9c98f3b495 is already attached to instances: 16b0498f-4cb8-491a-92cf-dac9890c7b4a Sep 04 01:10:58.342925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: action_args: {'server_id': '16b0498f-4cb8-491a-92cf-dac9890c7b4a', 'body': {'volumeAttachment': {'volumeId': '2b938702-b38c-4b5f-9d35-2a9c98f3b495'}}} Sep 04 01:10:58.342925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:10:58.342925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:10:58.342925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Sep 04 01:10:58.342925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Sep 04 01:10:58.342925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 04 01:10:58.342925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 04 01:10:58.342925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 04 01:10:58.342925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 04 01:10:58.342925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Sep 04 01:10:58.342925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Sep 04 01:10:58.342925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Sep 04 01:10:58.342925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Sep 04 01:10:58.342925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 2b938702-b38c-4b5f-9d35-2a9c98f3b495 is already attached to instances: 16b0498f-4cb8-491a-92cf-dac9890c7b4a Sep 04 01:10:58.342925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:10:58.342925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:10:58.342925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:10:58.342925 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:10:58.344065 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:10:58.344065 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:10:58.344065 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:10:58.344065 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:10:58.344065 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:10:58.344065 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:10:58.344065 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 04 01:10:58.344065 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 04 01:10:58.344065 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 04 01:10:58.344065 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 04 01:10:58.344065 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 04 01:10:58.344065 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 04 01:10:58.344065 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Sep 04 01:10:58.344065 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 04 01:10:58.344065 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 2b938702-b38c-4b5f-9d35-2a9c98f3b495 is already attached to instances: 16b0498f-4cb8-491a-92cf-dac9890c7b4a Sep 04 01:10:58.344065 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:10:58.344065 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:10:58.344065 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:10:58.344065 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:10:58.344065 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:10:58.345265 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:10:58.345265 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:10:58.345265 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:10:58.345265 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 2b938702-b38c-4b5f-9d35-2a9c98f3b495 is already attached to instances: 16b0498f-4cb8-491a-92cf-dac9890c7b4a Sep 04 01:10:58.345265 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:10:58.345265 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-dd5b46b8-7b11-4378-8cdb-bc49be89193c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Returning 400 to user: Invalid volume: volume 2b938702-b38c-4b5f-9d35-2a9c98f3b495 is already attached to instances: 16b0498f-4cb8-491a-92cf-dac9890c7b4a {{(pid=121036) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:10:58.345265 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-dd5b46b8-7b11-4378-8cdb-bc49be89193c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "POST /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.102080 Sep 04 01:10:58.345265 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 474/925] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Sep 4 01:10:58 2022] POST /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a/os-volume_attachments => generated 174 bytes in 103 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 04 01:10:58.411331 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-12360a86-ffb7-4e70-bc40-c5cd86d6c0e8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:58.411832 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-12360a86-ffb7-4e70-bc40-c5cd86d6c0e8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 16b0498f-4cb8-491a-92cf-dac9890c7b4a] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:58.421008 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-12360a86-ffb7-4e70-bc40-c5cd86d6c0e8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:58.421516 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-12360a86-ffb7-4e70-bc40-c5cd86d6c0e8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:58.421946 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-12360a86-ffb7-4e70-bc40-c5cd86d6c0e8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:58.422643 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-12360a86-ffb7-4e70-bc40-c5cd86d6c0e8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:58.423084 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-12360a86-ffb7-4e70-bc40-c5cd86d6c0e8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:58.423495 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-12360a86-ffb7-4e70-bc40-c5cd86d6c0e8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:58.508020 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.objects.instance [None req-12360a86-ffb7-4e70-bc40-c5cd86d6c0e8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lazy-loading 'services' on Instance uuid 16b0498f-4cb8-491a-92cf-dac9890c7b4a {{(pid=121037) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:10:58.735970 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-12360a86-ffb7-4e70-bc40-c5cd86d6c0e8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "DELETE /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a/os-volume_attachments/2b938702-b38c-4b5f-9d35-2a9c98f3b495" status: 202 len: 0 microversion: 2.1 time: 0.329535 Sep 04 01:10:58.736628 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 452/926] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Sep 4 01:10:58 2022] DELETE /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a/os-volume_attachments/2b938702-b38c-4b5f-9d35-2a9c98f3b495 => generated 0 bytes in 331 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 04 01:10:58.746054 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-df966ac2-5ea2-4944-bc70-5f3b68e0103a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:58.748481 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-df966ac2-5ea2-4944-bc70-5f3b68e0103a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 16b0498f-4cb8-491a-92cf-dac9890c7b4a] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:58.754795 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-df966ac2-5ea2-4944-bc70-5f3b68e0103a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:58.755349 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-df966ac2-5ea2-4944-bc70-5f3b68e0103a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:58.755845 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-df966ac2-5ea2-4944-bc70-5f3b68e0103a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:58.756478 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-df966ac2-5ea2-4944-bc70-5f3b68e0103a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:58.756929 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-df966ac2-5ea2-4944-bc70-5f3b68e0103a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:58.757373 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-df966ac2-5ea2-4944-bc70-5f3b68e0103a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:58.787666 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-df966ac2-5ea2-4944-bc70-5f3b68e0103a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045823 Sep 04 01:10:58.788216 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 475/927] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 4 01:10:58 2022] GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:10:59.801117 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-11cc3659-27dc-44c2-a77c-0514d49a7dd9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:10:59.803380 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-11cc3659-27dc-44c2-a77c-0514d49a7dd9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 16b0498f-4cb8-491a-92cf-dac9890c7b4a] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:10:59.809551 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-11cc3659-27dc-44c2-a77c-0514d49a7dd9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:59.810022 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-11cc3659-27dc-44c2-a77c-0514d49a7dd9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:59.810470 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-11cc3659-27dc-44c2-a77c-0514d49a7dd9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:59.811134 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-11cc3659-27dc-44c2-a77c-0514d49a7dd9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:10:59.811575 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-11cc3659-27dc-44c2-a77c-0514d49a7dd9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:10:59.811989 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-11cc3659-27dc-44c2-a77c-0514d49a7dd9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:10:59.841509 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-11cc3659-27dc-44c2-a77c-0514d49a7dd9 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045278 Sep 04 01:10:59.842048 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 453/928] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 4 01:10:59 2022] GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:11:00.853883 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-c11fb802-a572-464f-beba-1ce10dadae7c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:00.856741 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-c11fb802-a572-464f-beba-1ce10dadae7c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 16b0498f-4cb8-491a-92cf-dac9890c7b4a] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:00.863265 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c11fb802-a572-464f-beba-1ce10dadae7c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:00.863744 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c11fb802-a572-464f-beba-1ce10dadae7c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:00.864439 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c11fb802-a572-464f-beba-1ce10dadae7c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:00.865149 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c11fb802-a572-464f-beba-1ce10dadae7c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:00.865605 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c11fb802-a572-464f-beba-1ce10dadae7c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:00.866031 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c11fb802-a572-464f-beba-1ce10dadae7c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:00.894941 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-c11fb802-a572-464f-beba-1ce10dadae7c tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045062 Sep 04 01:11:00.895494 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 476/929] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 4 01:11:00 2022] GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:11:01.908307 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-8443da7d-bdce-4b7c-b7f5-d777721c9d73 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:01.910516 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-8443da7d-bdce-4b7c-b7f5-d777721c9d73 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 16b0498f-4cb8-491a-92cf-dac9890c7b4a] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:01.916940 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8443da7d-bdce-4b7c-b7f5-d777721c9d73 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:01.917467 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8443da7d-bdce-4b7c-b7f5-d777721c9d73 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:01.917908 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8443da7d-bdce-4b7c-b7f5-d777721c9d73 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:01.918589 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8443da7d-bdce-4b7c-b7f5-d777721c9d73 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:01.919045 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8443da7d-bdce-4b7c-b7f5-d777721c9d73 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:01.919677 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8443da7d-bdce-4b7c-b7f5-d777721c9d73 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:01.948616 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-8443da7d-bdce-4b7c-b7f5-d777721c9d73 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044924 Sep 04 01:11:01.949201 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 454/930] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 4 01:11:01 2022] GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:11:02.961942 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-583457a1-8a20-4f5e-9ece-c2f4b6557630 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:02.964077 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-583457a1-8a20-4f5e-9ece-c2f4b6557630 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 16b0498f-4cb8-491a-92cf-dac9890c7b4a] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:02.970499 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-583457a1-8a20-4f5e-9ece-c2f4b6557630 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:02.970976 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-583457a1-8a20-4f5e-9ece-c2f4b6557630 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:02.971407 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-583457a1-8a20-4f5e-9ece-c2f4b6557630 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:02.972087 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-583457a1-8a20-4f5e-9ece-c2f4b6557630 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:02.972588 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-583457a1-8a20-4f5e-9ece-c2f4b6557630 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:02.972949 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-583457a1-8a20-4f5e-9ece-c2f4b6557630 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:02.975341 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=81,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:11:03.005226 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-583457a1-8a20-4f5e-9ece-c2f4b6557630 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.047094 Sep 04 01:11:03.005789 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 477/931] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 4 01:11:02 2022] GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a/os-volume_attachments => generated 25 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 04 01:11:03.069105 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-146fac70-edfe-41f9-b593-74441a998da5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:03.069688 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-146fac70-edfe-41f9-b593-74441a998da5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 16b0498f-4cb8-491a-92cf-dac9890c7b4a] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:03.077094 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-146fac70-edfe-41f9-b593-74441a998da5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:03.077624 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-146fac70-edfe-41f9-b593-74441a998da5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:03.078058 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-146fac70-edfe-41f9-b593-74441a998da5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:03.078752 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-146fac70-edfe-41f9-b593-74441a998da5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:03.079198 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-146fac70-edfe-41f9-b593-74441a998da5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:03.079616 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-146fac70-edfe-41f9-b593-74441a998da5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:03.085035 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=92,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:11:03.106775 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-146fac70-edfe-41f9-b593-74441a998da5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 16b0498f-4cb8-491a-92cf-dac9890c7b4a] Going to try to terminate instance {{(pid=121037) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 04 01:11:03.355502 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-146fac70-edfe-41f9-b593-74441a998da5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "DELETE /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a" status: 204 len: 0 microversion: 2.1 time: 0.290032 Sep 04 01:11:03.356151 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 455/932] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 4 01:11:03 2022] DELETE /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a => generated 0 bytes in 292 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 04 01:11:03.365624 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-a7da7fb1-2f1c-4e73-b7a8-fa54f4de2c83 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:03.366256 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-a7da7fb1-2f1c-4e73-b7a8-fa54f4de2c83 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 16b0498f-4cb8-491a-92cf-dac9890c7b4a] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:03.372556 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7da7fb1-2f1c-4e73-b7a8-fa54f4de2c83 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:03.373664 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7da7fb1-2f1c-4e73-b7a8-fa54f4de2c83 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:03.374097 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7da7fb1-2f1c-4e73-b7a8-fa54f4de2c83 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:03.374906 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7da7fb1-2f1c-4e73-b7a8-fa54f4de2c83 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:03.375205 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7da7fb1-2f1c-4e73-b7a8-fa54f4de2c83 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:03.375622 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a7da7fb1-2f1c-4e73-b7a8-fa54f4de2c83 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:03.509238 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-a7da7fb1-2f1c-4e73-b7a8-fa54f4de2c83 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:03.518318 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-a7da7fb1-2f1c-4e73-b7a8-fa54f4de2c83 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a" status: 200 len: 1682 microversion: 2.1 time: 0.157237 Sep 04 01:11:03.518890 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 478/933] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:11:03 2022] GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a => generated 1682 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:11:04.549783 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-646b2d07-a923-431b-a2b8-62712457ed0a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:04.550500 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-646b2d07-a923-431b-a2b8-62712457ed0a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 16b0498f-4cb8-491a-92cf-dac9890c7b4a] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:04.556828 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-646b2d07-a923-431b-a2b8-62712457ed0a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:04.557332 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-646b2d07-a923-431b-a2b8-62712457ed0a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:04.557762 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-646b2d07-a923-431b-a2b8-62712457ed0a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:04.558439 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-646b2d07-a923-431b-a2b8-62712457ed0a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:04.558878 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-646b2d07-a923-431b-a2b8-62712457ed0a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:04.559297 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-646b2d07-a923-431b-a2b8-62712457ed0a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:04.686166 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-646b2d07-a923-431b-a2b8-62712457ed0a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:04.693836 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-646b2d07-a923-431b-a2b8-62712457ed0a tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a" status: 200 len: 1682 microversion: 2.1 time: 0.148923 Sep 04 01:11:04.694397 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 456/934] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:11:04 2022] GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a => generated 1682 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:11:05.717878 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-eea25589-e32a-4378-97d3-0247c84462ba tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:05.718523 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-eea25589-e32a-4378-97d3-0247c84462ba tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 16b0498f-4cb8-491a-92cf-dac9890c7b4a] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:05.725288 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-eea25589-e32a-4378-97d3-0247c84462ba tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:05.725839 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-eea25589-e32a-4378-97d3-0247c84462ba tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:05.726276 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-eea25589-e32a-4378-97d3-0247c84462ba tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:05.726971 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-eea25589-e32a-4378-97d3-0247c84462ba tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:05.727501 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-eea25589-e32a-4378-97d3-0247c84462ba tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:05.727989 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-eea25589-e32a-4378-97d3-0247c84462ba tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:05.868478 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-eea25589-e32a-4378-97d3-0247c84462ba tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:05.877556 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-eea25589-e32a-4378-97d3-0247c84462ba tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a" status: 200 len: 1682 microversion: 2.1 time: 0.164249 Sep 04 01:11:05.878295 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 479/935] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:11:05 2022] GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a => generated 1682 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:11:05.952185 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-03e07869-9d7a-4557-859a-5cffa003fe27 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:05.952760 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-03e07869-9d7a-4557-859a-5cffa003fe27 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:05.959219 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-03e07869-9d7a-4557-859a-5cffa003fe27 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:05.959778 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-03e07869-9d7a-4557-859a-5cffa003fe27 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:05.960279 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-03e07869-9d7a-4557-859a-5cffa003fe27 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:05.961068 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-03e07869-9d7a-4557-859a-5cffa003fe27 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:05.961479 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-03e07869-9d7a-4557-859a-5cffa003fe27 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:05.961890 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-03e07869-9d7a-4557-859a-5cffa003fe27 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:05.987127 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-03e07869-9d7a-4557-859a-5cffa003fe27 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b/ips" status: 200 len: 152 microversion: 2.20 time: 0.040140 Sep 04 01:11:05.987706 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 457/936] 10.222.0.42 () {60 vars in 1289 bytes} [Sun Sep 4 01:11:05 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b/ips => generated 152 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 04 01:11:06.115170 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-38459c74-d90d-4057-b173-8780f41ac25e req-914db43e-36ac-48e1-8537-985930114a83 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "16b0498f-4cb8-491a-92cf-dac9890c7b4a", "name": "network-vif-deleted", "tag": "2aad8c1f-5af7-4478-b2c4-782e5042cc6b"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:11:06.126128 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-38459c74-d90d-4057-b173-8780f41ac25e req-914db43e-36ac-48e1-8537-985930114a83 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:06.126755 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-38459c74-d90d-4057-b173-8780f41ac25e req-914db43e-36ac-48e1-8537-985930114a83 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:06.127280 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-38459c74-d90d-4057-b173-8780f41ac25e req-914db43e-36ac-48e1-8537-985930114a83 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:06.318144 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-38459c74-d90d-4057-b173-8780f41ac25e req-914db43e-36ac-48e1-8537-985930114a83 service nova] Creating event network-vif-deleted:2aad8c1f-5af7-4478-b2c4-782e5042cc6b for instance 16b0498f-4cb8-491a-92cf-dac9890c7b4a on hv-cinder-maste Sep 04 01:11:06.323634 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-38459c74-d90d-4057-b173-8780f41ac25e req-914db43e-36ac-48e1-8537-985930114a83 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.214671 Sep 04 01:11:06.324887 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 480/937] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:11:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:11:06.902629 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-65b79849-bb93-44d5-a5f5-70b7137fcc09 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:06.903454 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-65b79849-bb93-44d5-a5f5-70b7137fcc09 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 16b0498f-4cb8-491a-92cf-dac9890c7b4a] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:06.910319 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-65b79849-bb93-44d5-a5f5-70b7137fcc09 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:06.910813 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-65b79849-bb93-44d5-a5f5-70b7137fcc09 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:06.911275 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-65b79849-bb93-44d5-a5f5-70b7137fcc09 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:06.912146 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-65b79849-bb93-44d5-a5f5-70b7137fcc09 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:06.912606 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-65b79849-bb93-44d5-a5f5-70b7137fcc09 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:06.913033 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-65b79849-bb93-44d5-a5f5-70b7137fcc09 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:06.950433 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-65b79849-bb93-44d5-a5f5-70b7137fcc09 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:06.950966 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-65b79849-bb93-44d5-a5f5-70b7137fcc09 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:06.952165 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-65b79849-bb93-44d5-a5f5-70b7137fcc09 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:06.952877 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.objects.instance [None req-65b79849-bb93-44d5-a5f5-70b7137fcc09 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lazy-loading 'fault' on Instance uuid 16b0498f-4cb8-491a-92cf-dac9890c7b4a {{(pid=121037) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:11:06.997919 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-65b79849-bb93-44d5-a5f5-70b7137fcc09 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:07.006377 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-65b79849-bb93-44d5-a5f5-70b7137fcc09 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a" status: 200 len: 1331 microversion: 2.1 time: 0.108015 Sep 04 01:11:07.006966 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 458/938] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:11:06 2022] GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a => generated 1331 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:11:08.031534 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b0e0f5a1-62bb-425d-b988-da62ca32b5a4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:08.033117 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-b0e0f5a1-62bb-425d-b988-da62ca32b5a4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 16b0498f-4cb8-491a-92cf-dac9890c7b4a] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:08.039507 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b0e0f5a1-62bb-425d-b988-da62ca32b5a4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:08.039986 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b0e0f5a1-62bb-425d-b988-da62ca32b5a4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:08.040456 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b0e0f5a1-62bb-425d-b988-da62ca32b5a4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:08.041960 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b0e0f5a1-62bb-425d-b988-da62ca32b5a4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:08.041960 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b0e0f5a1-62bb-425d-b988-da62ca32b5a4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:08.041960 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b0e0f5a1-62bb-425d-b988-da62ca32b5a4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:08.055998 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.wsgi [None req-b0e0f5a1-62bb-425d-b988-da62ca32b5a4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] HTTP exception thrown: Instance 16b0498f-4cb8-491a-92cf-dac9890c7b4a could not be found. Sep 04 01:11:08.057699 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b0e0f5a1-62bb-425d-b988-da62ca32b5a4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Request method failure captured: Sep 04 01:11:08.057699 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: request: GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a HTTP/1.1 Sep 04 01:11:08.057699 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept: application/json Sep 04 01:11:08.057699 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept-Encoding: identity Sep 04 01:11:08.057699 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Connection: close Sep 04 01:11:08.057699 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Length: 0 Sep 04 01:11:08.057699 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Type: application/json Sep 04 01:11:08.057699 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Host: 10.222.0.42 Sep 04 01:11:08.057699 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Openstack-System-Scope: None Sep 04 01:11:08.057699 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: User-Agent: python-urllib3/1.26.12 Sep 04 01:11:08.057699 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Auth-Token: gAAAAABjE_nLdulKsCh7rP69EyY8StqBiRnQNFlCX5VoWZ9XzP_YTEis0NNuYGIELGjVYA3FFaOCC9PLYuhPddV3AgDhoAQw_S2o9MK0q1A57ehVT3gYS6mSkklOOLScx3a8r0TUFxeOMhMymWFJt03FSLp4K4nMUe8FttsVqGRsky_ik39T3Zg Sep 04 01:11:08.057699 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Id: None Sep 04 01:11:08.057699 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Name: None Sep 04 01:11:08.057699 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Identity-Status: Confirmed Sep 04 01:11:08.057699 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Is-Admin-Project: True Sep 04 01:11:08.057699 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Id: default Sep 04 01:11:08.057699 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Name: Default Sep 04 01:11:08.057699 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Id: 287d111f08ac4b05bd5adec9c40d0620 Sep 04 01:11:08.057699 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Name: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:11:08.057699 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Role: reader,member Sep 04 01:11:08.057699 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Roles: reader,member Sep 04 01:11:08.059063 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_287d111f08ac4b05bd5adec9c40d0620", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:11:08.059063 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:11:08.059063 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Id: 287d111f08ac4b05bd5adec9c40d0620 Sep 04 01:11:08.059063 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:11:08.059063 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User: tempest-AttachVolumeNegativeTest-825474236-project Sep 04 01:11:08.059063 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Id: default Sep 04 01:11:08.059063 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Name: Default Sep 04 01:11:08.059063 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Id: f5e4723258c64ce5885154dded2d3d43 Sep 04 01:11:08.059063 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Name: tempest-AttachVolumeNegativeTest-825474236-project Sep 04 01:11:08.059063 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: method: > Sep 04 01:11:08.059063 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: exception: Instance 16b0498f-4cb8-491a-92cf-dac9890c7b4a could not be found. Sep 04 01:11:08.059063 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: action_args: {'id': '16b0498f-4cb8-491a-92cf-dac9890c7b4a'} Sep 04 01:11:08.060031 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:11:08.060031 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:11:08.060031 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:11:08.060031 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:11:08.060031 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:11:08.060031 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:11:08.060031 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:11:08.060031 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:11:08.060031 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:11:08.060031 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:11:08.060031 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 16b0498f-4cb8-491a-92cf-dac9890c7b4a could not be found. Sep 04 01:11:08.060031 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:11:08.060031 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:11:08.060031 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:11:08.060031 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:11:08.060031 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:11:08.060031 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:11:08.060031 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:11:08.060031 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:11:08.060031 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:11:08.061482 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:11:08.061482 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 04 01:11:08.061482 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 04 01:11:08.061482 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:11:08.061482 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:11:08.061482 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:11:08.061482 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:11:08.061482 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 16b0498f-4cb8-491a-92cf-dac9890c7b4a could not be found. Sep 04 01:11:08.061482 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:11:08.061482 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:11:08.061482 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:11:08.061482 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:11:08.061482 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:11:08.061482 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:11:08.061482 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:11:08.061482 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:11:08.061482 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 16b0498f-4cb8-491a-92cf-dac9890c7b4a could not be found. Sep 04 01:11:08.061482 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:11:08.062543 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b0e0f5a1-62bb-425d-b988-da62ca32b5a4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Returning 404 to user: Instance 16b0498f-4cb8-491a-92cf-dac9890c7b4a could not be found. {{(pid=121036) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:11:08.062543 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-b0e0f5a1-62bb-425d-b988-da62ca32b5a4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a" status: 404 len: 111 microversion: 2.1 time: 0.032315 Sep 04 01:11:08.062543 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 481/939] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:11:08 2022] GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 04 01:11:08.070644 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-39d32175-10d4-415b-97fc-f9c1ae312301 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '.version_select at 0x7fc10c4ceee0>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:08.148409 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-39d32175-10d4-415b-97fc-f9c1ae312301 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-510224483" status: 202 len: 0 microversion: 2.1 time: 0.084194 Sep 04 01:11:08.148994 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 459/940] 10.222.0.42 () {58 vars in 1218 bytes} [Sun Sep 4 01:11:08 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-510224483 => generated 0 bytes in 85 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 04 01:11:08.170850 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-8ff3cbf7-1b3c-484e-bae9-f405b45ef895 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:11:08.171405 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-8ff3cbf7-1b3c-484e-bae9-f405b45ef895 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:08.178147 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8ff3cbf7-1b3c-484e-bae9-f405b45ef895 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:08.178637 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8ff3cbf7-1b3c-484e-bae9-f405b45ef895 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:08.179079 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8ff3cbf7-1b3c-484e-bae9-f405b45ef895 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:08.179780 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8ff3cbf7-1b3c-484e-bae9-f405b45ef895 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:08.180228 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8ff3cbf7-1b3c-484e-bae9-f405b45ef895 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:08.180657 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8ff3cbf7-1b3c-484e-bae9-f405b45ef895 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:08.205171 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.objects.instance [None req-8ff3cbf7-1b3c-484e-bae9-f405b45ef895 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lazy-loading 'flavor' on Instance uuid da1d8cad-5ea0-48fc-a5af-81eec3d1e94b {{(pid=121036) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:11:08.624272 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-8ff3cbf7-1b3c-484e-bae9-f405b45ef895 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "POST /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b/action" status: 202 len: 0 microversion: 2.20 time: 0.457180 Sep 04 01:11:08.624788 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 482/941] 10.222.0.42 () {62 vars in 1319 bytes} [Sun Sep 4 01:11:08 2022] POST /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b/action => generated 0 bytes in 459 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Sep 04 01:11:08.632975 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-26702bd0-da30-4e86-934e-a5ff82fbbd36 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:08.633923 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-26702bd0-da30-4e86-934e-a5ff82fbbd36 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:08.640186 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-26702bd0-da30-4e86-934e-a5ff82fbbd36 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:08.640678 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-26702bd0-da30-4e86-934e-a5ff82fbbd36 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:08.641181 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-26702bd0-da30-4e86-934e-a5ff82fbbd36 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:08.641887 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-26702bd0-da30-4e86-934e-a5ff82fbbd36 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:08.642336 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-26702bd0-da30-4e86-934e-a5ff82fbbd36 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:08.642758 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-26702bd0-da30-4e86-934e-a5ff82fbbd36 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:08.793697 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-26702bd0-da30-4e86-934e-a5ff82fbbd36 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:08.807990 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-26702bd0-da30-4e86-934e-a5ff82fbbd36 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:08.807990 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-26702bd0-da30-4e86-934e-a5ff82fbbd36 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:08.807990 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-26702bd0-da30-4e86-934e-a5ff82fbbd36 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1722 microversion: 2.20 time: 0.176189 Sep 04 01:11:08.808321 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 460/942] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:08 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1722 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:11:08.990156 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-39214263-89bc-49bb-bbe2-cd00f1329791 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Action: 'create', calling method: .version_select at 0x7fc10c8199d0>, body: {"keypair": {"name": "tempest-keypair-1092737247", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDcj/qF57B3H1lq6MN5F0l/2uWgxJTZgEMkOA7hvXp5WVo+c2pDRC7kZxRd0hlYPEaj/xcGPjgxLHVKY1P4Y4IDO5qDtklQwp1AnE+A9RQvwZjgpEN/b5IHQXbSHtDd2fg=="}} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:11:09.008498 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-39214263-89bc-49bb-bbe2-cd00f1329791 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Getting quotas for project 287d111f08ac4b05bd5adec9c40d0620. Resources: {'key_pairs'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 04 01:11:09.012234 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-39214263-89bc-49bb-bbe2-cd00f1329791 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Getting quotas for user f5e4723258c64ce5885154dded2d3d43 and project 287d111f08ac4b05bd5adec9c40d0620. Resources: {'key_pairs'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 04 01:11:09.090576 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-39214263-89bc-49bb-bbe2-cd00f1329791 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.106557 Sep 04 01:11:09.091203 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 483/943] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Sep 4 01:11:08 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:11:09.831826 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-12c59060-4fd8-4e6d-bb07-5fa6d0c52e1e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:09.832534 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-12c59060-4fd8-4e6d-bb07-5fa6d0c52e1e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:09.838981 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-12c59060-4fd8-4e6d-bb07-5fa6d0c52e1e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:09.839449 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-12c59060-4fd8-4e6d-bb07-5fa6d0c52e1e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:09.839875 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-12c59060-4fd8-4e6d-bb07-5fa6d0c52e1e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:09.840550 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-12c59060-4fd8-4e6d-bb07-5fa6d0c52e1e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:09.840991 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-12c59060-4fd8-4e6d-bb07-5fa6d0c52e1e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:09.841438 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-12c59060-4fd8-4e6d-bb07-5fa6d0c52e1e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:09.984736 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-12c59060-4fd8-4e6d-bb07-5fa6d0c52e1e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:09.993530 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-12c59060-4fd8-4e6d-bb07-5fa6d0c52e1e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:09.995099 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-12c59060-4fd8-4e6d-bb07-5fa6d0c52e1e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:09.996386 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-12c59060-4fd8-4e6d-bb07-5fa6d0c52e1e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1722 microversion: 2.20 time: 0.168377 Sep 04 01:11:09.997008 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 461/944] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:09 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1722 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:11:11.023906 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-589845c1-6e54-4cb1-abe4-eef47d55ada2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:11.024654 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-589845c1-6e54-4cb1-abe4-eef47d55ada2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:11.031640 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-589845c1-6e54-4cb1-abe4-eef47d55ada2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:11.032475 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-589845c1-6e54-4cb1-abe4-eef47d55ada2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:11.032924 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-589845c1-6e54-4cb1-abe4-eef47d55ada2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:11.034902 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-589845c1-6e54-4cb1-abe4-eef47d55ada2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:11.034902 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-589845c1-6e54-4cb1-abe4-eef47d55ada2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:11.034902 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-589845c1-6e54-4cb1-abe4-eef47d55ada2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:11.271068 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-589845c1-6e54-4cb1-abe4-eef47d55ada2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:11.279891 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-589845c1-6e54-4cb1-abe4-eef47d55ada2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:11.282024 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-589845c1-6e54-4cb1-abe4-eef47d55ada2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:11.283183 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-589845c1-6e54-4cb1-abe4-eef47d55ada2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1722 microversion: 2.20 time: 0.264167 Sep 04 01:11:11.283884 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 484/945] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:11 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1722 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:11:11.619143 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-0d25af1a-9bf0-4302-a934-30a7b8a3d555 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-436321442", "imageRef": "a11315c3-ce81-478b-91e4-5353f189dfa7", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "d7f39628-666e-4480-8bee-49de183d5e09"}], "security_groups": [{"name": "tempest-securitygroup--637612000"}], "key_name": "tempest-keypair-1092737247", "user_data": "}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:11:11.723172 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.network.neutron [None req-0d25af1a-9bf0-4302-a934-30a7b8a3d555 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] validate_networks() for [('d7f39628-666e-4480-8bee-49de183d5e09', None, None, None, None, None)] {{(pid=121037) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 04 01:11:12.208991 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-0d25af1a-9bf0-4302-a934-30a7b8a3d555 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Flavor limits 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 04 01:11:12.211806 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-0d25af1a-9bf0-4302-a934-30a7b8a3d555 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Image limits 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 04 01:11:12.211806 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-0d25af1a-9bf0-4302-a934-30a7b8a3d555 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Flavor pref 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 04 01:11:12.211806 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-0d25af1a-9bf0-4302-a934-30a7b8a3d555 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Image pref 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 04 01:11:12.211806 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-0d25af1a-9bf0-4302-a934-30a7b8a3d555 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 04 01:11:12.219358 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0d25af1a-9bf0-4302-a934-30a7b8a3d555 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:12.219837 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0d25af1a-9bf0-4302-a934-30a7b8a3d555 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:12.220272 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0d25af1a-9bf0-4302-a934-30a7b8a3d555 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:12.221000 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0d25af1a-9bf0-4302-a934-30a7b8a3d555 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:12.221558 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0d25af1a-9bf0-4302-a934-30a7b8a3d555 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:12.221984 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0d25af1a-9bf0-4302-a934-30a7b8a3d555 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:12.238332 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.quota [None req-0d25af1a-9bf0-4302-a934-30a7b8a3d555 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Getting quotas for project 287d111f08ac4b05bd5adec9c40d0620. Resources: {'ram', 'instances', 'cores'} {{(pid=121037) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 04 01:11:12.242654 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.quota [None req-0d25af1a-9bf0-4302-a934-30a7b8a3d555 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Getting quotas for user f5e4723258c64ce5885154dded2d3d43 and project 287d111f08ac4b05bd5adec9c40d0620. Resources: {'ram', 'instances', 'cores'} {{(pid=121037) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 04 01:11:12.254133 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-0d25af1a-9bf0-4302-a934-30a7b8a3d555 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Going to run 1 instances... {{(pid=121037) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 04 01:11:12.257725 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-0d25af1a-9bf0-4302-a934-30a7b8a3d555 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 2718d3b6-c71c-4ae1-9544-0f8e4a43d989] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a11315c3-ce81-478b-91e4-5353f189dfa7',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121037) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 04 01:11:12.309712 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-0d503ac7-8a92-44de-8309-b274b65fe23e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:12.309712 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-0d503ac7-8a92-44de-8309-b274b65fe23e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:12.316217 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0d503ac7-8a92-44de-8309-b274b65fe23e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:12.316691 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0d503ac7-8a92-44de-8309-b274b65fe23e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:12.317150 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0d503ac7-8a92-44de-8309-b274b65fe23e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:12.317833 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0d503ac7-8a92-44de-8309-b274b65fe23e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:12.318279 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0d503ac7-8a92-44de-8309-b274b65fe23e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:12.318697 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0d503ac7-8a92-44de-8309-b274b65fe23e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:12.401218 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-0d25af1a-9bf0-4302-a934-30a7b8a3d555 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.786336 Sep 04 01:11:12.402202 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 462/946] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 4 01:11:11 2022] POST /compute/v2.1/servers => generated 401 bytes in 788 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 04 01:11:12.415019 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-50523877-af29-4370-9cce-3164017fcf17 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:12.416156 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-50523877-af29-4370-9cce-3164017fcf17 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 2718d3b6-c71c-4ae1-9544-0f8e4a43d989] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:12.465287 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-0d503ac7-8a92-44de-8309-b274b65fe23e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:12.473777 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-0d503ac7-8a92-44de-8309-b274b65fe23e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:12.476064 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-0d503ac7-8a92-44de-8309-b274b65fe23e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:12.476378 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-0d503ac7-8a92-44de-8309-b274b65fe23e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1722 microversion: 2.20 time: 0.171628 Sep 04 01:11:12.480663 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 485/947] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:12 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1722 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:11:12.497275 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-50523877-af29-4370-9cce-3164017fcf17 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:12.505468 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-50523877-af29-4370-9cce-3164017fcf17 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989" status: 200 len: 1241 microversion: 2.1 time: 0.095812 Sep 04 01:11:12.506158 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 463/948] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:11:12 2022] GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989 => generated 1241 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:11:13.504760 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-dc5770c2-6613-44d1-a1c7-c9064ca313dd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:13.504760 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-dc5770c2-6613-44d1-a1c7-c9064ca313dd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:13.510880 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dc5770c2-6613-44d1-a1c7-c9064ca313dd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:13.511386 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dc5770c2-6613-44d1-a1c7-c9064ca313dd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:13.511822 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dc5770c2-6613-44d1-a1c7-c9064ca313dd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:13.512504 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dc5770c2-6613-44d1-a1c7-c9064ca313dd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:13.512951 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dc5770c2-6613-44d1-a1c7-c9064ca313dd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:13.513401 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dc5770c2-6613-44d1-a1c7-c9064ca313dd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:13.531833 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-d4624ae7-d917-4b90-a2ef-7a0f363f0437 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:13.533358 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-d4624ae7-d917-4b90-a2ef-7a0f363f0437 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 2718d3b6-c71c-4ae1-9544-0f8e4a43d989] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:13.542013 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d4624ae7-d917-4b90-a2ef-7a0f363f0437 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:13.542627 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d4624ae7-d917-4b90-a2ef-7a0f363f0437 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:13.543712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d4624ae7-d917-4b90-a2ef-7a0f363f0437 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:13.545249 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d4624ae7-d917-4b90-a2ef-7a0f363f0437 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:13.546327 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d4624ae7-d917-4b90-a2ef-7a0f363f0437 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:13.547252 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d4624ae7-d917-4b90-a2ef-7a0f363f0437 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:13.623915 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-d4624ae7-d917-4b90-a2ef-7a0f363f0437 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:13.632324 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-d4624ae7-d917-4b90-a2ef-7a0f363f0437 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989" status: 200 len: 1245 microversion: 2.1 time: 0.107744 Sep 04 01:11:13.633181 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 464/949] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:11:13 2022] GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989 => generated 1245 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:11:13.673120 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-dc5770c2-6613-44d1-a1c7-c9064ca313dd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:13.682100 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-dc5770c2-6613-44d1-a1c7-c9064ca313dd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:13.683966 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-dc5770c2-6613-44d1-a1c7-c9064ca313dd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:13.685354 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-dc5770c2-6613-44d1-a1c7-c9064ca313dd tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1722 microversion: 2.20 time: 0.186000 Sep 04 01:11:13.686547 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 486/950] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:13 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1722 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:11:14.655922 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-b6268caf-aef3-4eef-889c-0ce43cc9c21d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:14.656623 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-b6268caf-aef3-4eef-889c-0ce43cc9c21d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 2718d3b6-c71c-4ae1-9544-0f8e4a43d989] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:14.663531 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b6268caf-aef3-4eef-889c-0ce43cc9c21d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:14.664092 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b6268caf-aef3-4eef-889c-0ce43cc9c21d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:14.664528 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b6268caf-aef3-4eef-889c-0ce43cc9c21d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:14.665245 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b6268caf-aef3-4eef-889c-0ce43cc9c21d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:14.665693 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b6268caf-aef3-4eef-889c-0ce43cc9c21d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:14.666106 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b6268caf-aef3-4eef-889c-0ce43cc9c21d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:14.711395 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-0601574b-e336-46fb-94c2-47851205c92a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:14.712013 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-0601574b-e336-46fb-94c2-47851205c92a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:14.731272 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_api:SELECT=45,nova_api:INSERT=1 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:11:14.733063 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-b6268caf-aef3-4eef-889c-0ce43cc9c21d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:14.738432 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0601574b-e336-46fb-94c2-47851205c92a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:14.739845 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0601574b-e336-46fb-94c2-47851205c92a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:14.740246 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0601574b-e336-46fb-94c2-47851205c92a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:14.741273 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0601574b-e336-46fb-94c2-47851205c92a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:14.741887 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0601574b-e336-46fb-94c2-47851205c92a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:14.742307 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0601574b-e336-46fb-94c2-47851205c92a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:14.744142 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-b6268caf-aef3-4eef-889c-0ce43cc9c21d tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989" status: 200 len: 1293 microversion: 2.1 time: 0.092057 Sep 04 01:11:14.744823 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 465/951] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:11:14 2022] GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989 => generated 1293 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:11:14.885808 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-0601574b-e336-46fb-94c2-47851205c92a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:14.894249 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-0601574b-e336-46fb-94c2-47851205c92a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:14.895699 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-0601574b-e336-46fb-94c2-47851205c92a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:14.896865 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-0601574b-e336-46fb-94c2-47851205c92a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1722 microversion: 2.20 time: 0.189441 Sep 04 01:11:14.897458 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 487/952] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:14 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1722 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:11:15.769466 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-43259064-6c49-42cc-b7be-e7a5320cddbc tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:15.770085 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-43259064-6c49-42cc-b7be-e7a5320cddbc tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 2718d3b6-c71c-4ae1-9544-0f8e4a43d989] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:15.776726 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-43259064-6c49-42cc-b7be-e7a5320cddbc tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:15.777516 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-43259064-6c49-42cc-b7be-e7a5320cddbc tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:15.778351 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-43259064-6c49-42cc-b7be-e7a5320cddbc tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:15.779930 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-43259064-6c49-42cc-b7be-e7a5320cddbc tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:15.780539 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-43259064-6c49-42cc-b7be-e7a5320cddbc tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:15.782045 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-43259064-6c49-42cc-b7be-e7a5320cddbc tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:15.923950 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-c7f58825-5468-4449-83e2-a1696953915b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:15.924670 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-c7f58825-5468-4449-83e2-a1696953915b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:15.932106 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c7f58825-5468-4449-83e2-a1696953915b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:15.932767 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c7f58825-5468-4449-83e2-a1696953915b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:15.933491 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c7f58825-5468-4449-83e2-a1696953915b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:15.934463 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c7f58825-5468-4449-83e2-a1696953915b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:15.935131 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c7f58825-5468-4449-83e2-a1696953915b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:15.935947 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c7f58825-5468-4449-83e2-a1696953915b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:15.975663 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-43259064-6c49-42cc-b7be-e7a5320cddbc tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:15.986005 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-43259064-6c49-42cc-b7be-e7a5320cddbc tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989" status: 200 len: 1366 microversion: 2.1 time: 0.222061 Sep 04 01:11:15.987360 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 466/953] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:11:15 2022] GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989 => generated 1366 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:11:16.139434 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-c7f58825-5468-4449-83e2-a1696953915b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:16.147886 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-c7f58825-5468-4449-83e2-a1696953915b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:16.149863 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-c7f58825-5468-4449-83e2-a1696953915b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:16.155477 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-c7f58825-5468-4449-83e2-a1696953915b tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1722 microversion: 2.20 time: 0.232317 Sep 04 01:11:16.155477 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 488/954] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:15 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1722 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:11:16.814902 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-69a5ab0d-8198-4040-9ee0-d03939347395 req-05afcdb5-ead2-48f6-b6dc-24f0302188f9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2718d3b6-c71c-4ae1-9544-0f8e4a43d989", "tag": "a684da0f-7a2f-4974-8aa3-30875caa5a0e"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:11:16.832935 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-69a5ab0d-8198-4040-9ee0-d03939347395 req-05afcdb5-ead2-48f6-b6dc-24f0302188f9 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:16.833502 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-69a5ab0d-8198-4040-9ee0-d03939347395 req-05afcdb5-ead2-48f6-b6dc-24f0302188f9 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:16.834223 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-69a5ab0d-8198-4040-9ee0-d03939347395 req-05afcdb5-ead2-48f6-b6dc-24f0302188f9 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:16.852940 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-69a5ab0d-8198-4040-9ee0-d03939347395 req-05afcdb5-ead2-48f6-b6dc-24f0302188f9 service nova] Creating event network-changed:a684da0f-7a2f-4974-8aa3-30875caa5a0e for instance 2718d3b6-c71c-4ae1-9544-0f8e4a43d989 on hv-cinder-maste Sep 04 01:11:16.858092 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-69a5ab0d-8198-4040-9ee0-d03939347395 req-05afcdb5-ead2-48f6-b6dc-24f0302188f9 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.059351 Sep 04 01:11:16.858697 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 467/955] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:11:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:11:17.015101 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-ff1495bd-dcbb-436a-881d-75f9a4d235b4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:17.015712 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-ff1495bd-dcbb-436a-881d-75f9a4d235b4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 2718d3b6-c71c-4ae1-9544-0f8e4a43d989] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:17.022459 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ff1495bd-dcbb-436a-881d-75f9a4d235b4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:17.022945 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ff1495bd-dcbb-436a-881d-75f9a4d235b4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:17.023384 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ff1495bd-dcbb-436a-881d-75f9a4d235b4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:17.024085 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ff1495bd-dcbb-436a-881d-75f9a4d235b4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:17.024527 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ff1495bd-dcbb-436a-881d-75f9a4d235b4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:17.024943 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ff1495bd-dcbb-436a-881d-75f9a4d235b4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:17.155090 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-f85c4a28-314d-4898-b04f-99d4cc5c5303 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:17.155851 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-f85c4a28-314d-4898-b04f-99d4cc5c5303 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:17.163033 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f85c4a28-314d-4898-b04f-99d4cc5c5303 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:17.167179 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f85c4a28-314d-4898-b04f-99d4cc5c5303 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:17.167796 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ff1495bd-dcbb-436a-881d-75f9a4d235b4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:17.167914 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f85c4a28-314d-4898-b04f-99d4cc5c5303 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:17.167914 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f85c4a28-314d-4898-b04f-99d4cc5c5303 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:17.167914 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f85c4a28-314d-4898-b04f-99d4cc5c5303 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:17.167914 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f85c4a28-314d-4898-b04f-99d4cc5c5303 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:17.177005 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-ff1495bd-dcbb-436a-881d-75f9a4d235b4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989" status: 200 len: 1366 microversion: 2.1 time: 0.166827 Sep 04 01:11:17.178140 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 489/956] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:11:17 2022] GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989 => generated 1366 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:11:17.207788 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-cd4b50eb-95b1-450f-8934-a0e48b17e4b7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:17.207788 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-cd4b50eb-95b1-450f-8934-a0e48b17e4b7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:17.207788 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cd4b50eb-95b1-450f-8934-a0e48b17e4b7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:17.207788 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cd4b50eb-95b1-450f-8934-a0e48b17e4b7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:17.208542 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cd4b50eb-95b1-450f-8934-a0e48b17e4b7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:17.208542 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cd4b50eb-95b1-450f-8934-a0e48b17e4b7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:17.208542 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cd4b50eb-95b1-450f-8934-a0e48b17e4b7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:17.208542 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cd4b50eb-95b1-450f-8934-a0e48b17e4b7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:17.381814 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-f85c4a28-314d-4898-b04f-99d4cc5c5303 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219/ips" status: 200 len: 145 microversion: 2.1 time: 0.232003 Sep 04 01:11:17.382854 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 468/957] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Sep 4 01:11:17 2022] GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219/ips => generated 145 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:11:17.389529 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-cd4b50eb-95b1-450f-8934-a0e48b17e4b7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:17.400902 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-cd4b50eb-95b1-450f-8934-a0e48b17e4b7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:17.402650 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-cd4b50eb-95b1-450f-8934-a0e48b17e4b7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:17.403821 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-cd4b50eb-95b1-450f-8934-a0e48b17e4b7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1722 microversion: 2.20 time: 0.224625 Sep 04 01:11:17.404438 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 490/958] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:17 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1722 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:11:18.200226 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-98a6600d-c88f-447f-8d39-148a474cac78 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:18.200761 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-98a6600d-c88f-447f-8d39-148a474cac78 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 2718d3b6-c71c-4ae1-9544-0f8e4a43d989] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:18.207321 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-98a6600d-c88f-447f-8d39-148a474cac78 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:18.207796 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-98a6600d-c88f-447f-8d39-148a474cac78 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:18.208224 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-98a6600d-c88f-447f-8d39-148a474cac78 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:18.208900 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-98a6600d-c88f-447f-8d39-148a474cac78 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:18.209377 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-98a6600d-c88f-447f-8d39-148a474cac78 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:18.209802 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-98a6600d-c88f-447f-8d39-148a474cac78 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:18.342916 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-98a6600d-c88f-447f-8d39-148a474cac78 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:18.350653 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-98a6600d-c88f-447f-8d39-148a474cac78 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989" status: 200 len: 1366 microversion: 2.1 time: 0.154352 Sep 04 01:11:18.351422 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 469/959] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:11:18 2022] GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989 => generated 1366 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:11:18.430866 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-d13e1618-d3ad-4777-9761-7992be0cb344 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:18.432087 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-d13e1618-d3ad-4777-9761-7992be0cb344 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:18.439744 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d13e1618-d3ad-4777-9761-7992be0cb344 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:18.440255 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d13e1618-d3ad-4777-9761-7992be0cb344 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:18.440682 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d13e1618-d3ad-4777-9761-7992be0cb344 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:18.441424 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d13e1618-d3ad-4777-9761-7992be0cb344 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:18.441883 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d13e1618-d3ad-4777-9761-7992be0cb344 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:18.442311 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d13e1618-d3ad-4777-9761-7992be0cb344 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:18.571979 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-d13e1618-d3ad-4777-9761-7992be0cb344 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:18.580619 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-d13e1618-d3ad-4777-9761-7992be0cb344 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:18.582085 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-d13e1618-d3ad-4777-9761-7992be0cb344 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:18.583294 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-d13e1618-d3ad-4777-9761-7992be0cb344 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1722 microversion: 2.20 time: 0.156668 Sep 04 01:11:18.583856 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 491/960] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:18 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1722 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:11:19.081211 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-9433e7c5-e4ac-4e0b-97ea-c276b2cbede3 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "d0664352-6a12-4f27-b283-95de87dddc60"}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:11:19.083571 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-9433e7c5-e4ac-4e0b-97ea-c276b2cbede3 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:19.085559 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_api:SELECT=65,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:11:19.090200 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9433e7c5-e4ac-4e0b-97ea-c276b2cbede3 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:19.090680 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9433e7c5-e4ac-4e0b-97ea-c276b2cbede3 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:19.091112 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9433e7c5-e4ac-4e0b-97ea-c276b2cbede3 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:19.092558 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9433e7c5-e4ac-4e0b-97ea-c276b2cbede3 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:19.093179 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9433e7c5-e4ac-4e0b-97ea-c276b2cbede3 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:19.093609 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9433e7c5-e4ac-4e0b-97ea-c276b2cbede3 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:19.380230 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b9408a6d-8e1f-4a92-b02b-71772f5436c8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:19.380836 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-b9408a6d-8e1f-4a92-b02b-71772f5436c8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 2718d3b6-c71c-4ae1-9544-0f8e4a43d989] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:19.382768 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell0:SELECT=4 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:11:19.387616 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b9408a6d-8e1f-4a92-b02b-71772f5436c8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:19.388097 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b9408a6d-8e1f-4a92-b02b-71772f5436c8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:19.388586 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b9408a6d-8e1f-4a92-b02b-71772f5436c8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:19.389372 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b9408a6d-8e1f-4a92-b02b-71772f5436c8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:19.389831 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b9408a6d-8e1f-4a92-b02b-71772f5436c8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:19.391024 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b9408a6d-8e1f-4a92-b02b-71772f5436c8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:19.525641 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-b9408a6d-8e1f-4a92-b02b-71772f5436c8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:19.534042 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-b9408a6d-8e1f-4a92-b02b-71772f5436c8 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989" status: 200 len: 1366 microversion: 2.1 time: 0.157835 Sep 04 01:11:19.534621 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 492/961] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:11:19 2022] GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989 => generated 1366 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:11:19.607947 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-50c1aae2-f9e7-49e9-98ab-a2e945800a39 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:19.608548 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-50c1aae2-f9e7-49e9-98ab-a2e945800a39 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:19.614732 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-50c1aae2-f9e7-49e9-98ab-a2e945800a39 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:19.615247 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-50c1aae2-f9e7-49e9-98ab-a2e945800a39 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:19.616746 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-50c1aae2-f9e7-49e9-98ab-a2e945800a39 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:19.616746 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-50c1aae2-f9e7-49e9-98ab-a2e945800a39 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:19.616933 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-50c1aae2-f9e7-49e9-98ab-a2e945800a39 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:19.617232 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-50c1aae2-f9e7-49e9-98ab-a2e945800a39 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:19.751135 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-50c1aae2-f9e7-49e9-98ab-a2e945800a39 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:19.759794 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-50c1aae2-f9e7-49e9-98ab-a2e945800a39 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:19.761377 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-50c1aae2-f9e7-49e9-98ab-a2e945800a39 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:19.762616 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-50c1aae2-f9e7-49e9-98ab-a2e945800a39 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1743 microversion: 2.20 time: 0.158761 Sep 04 01:11:19.763436 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 493/962] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:19 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1743 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:11:19.830928 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-9433e7c5-e4ac-4e0b-97ea-c276b2cbede3 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "POST /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.753446 Sep 04 01:11:19.831664 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 470/963] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Sep 4 01:11:19 2022] POST /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219/os-volume_attachments => generated 194 bytes in 756 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:11:20.556950 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-51b13de7-659c-469c-aa34-0ff998c97120 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:20.557587 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-51b13de7-659c-469c-aa34-0ff998c97120 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 2718d3b6-c71c-4ae1-9544-0f8e4a43d989] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:20.564663 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-51b13de7-659c-469c-aa34-0ff998c97120 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:20.565241 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-51b13de7-659c-469c-aa34-0ff998c97120 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:20.565719 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-51b13de7-659c-469c-aa34-0ff998c97120 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:20.566413 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-51b13de7-659c-469c-aa34-0ff998c97120 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:20.566910 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-51b13de7-659c-469c-aa34-0ff998c97120 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:20.567321 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-51b13de7-659c-469c-aa34-0ff998c97120 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:20.571401 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:11:20.696290 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-51b13de7-659c-469c-aa34-0ff998c97120 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:20.704482 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-51b13de7-659c-469c-aa34-0ff998c97120 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989" status: 200 len: 1366 microversion: 2.1 time: 0.151282 Sep 04 01:11:20.705040 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 494/964] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:11:20 2022] GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989 => generated 1366 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:11:20.790350 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-5270f7f9-374f-485b-b800-e4e2f6f9a2bf tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:20.791039 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-5270f7f9-374f-485b-b800-e4e2f6f9a2bf tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:20.797348 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5270f7f9-374f-485b-b800-e4e2f6f9a2bf tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:20.797824 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5270f7f9-374f-485b-b800-e4e2f6f9a2bf tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:20.798255 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5270f7f9-374f-485b-b800-e4e2f6f9a2bf tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:20.798938 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5270f7f9-374f-485b-b800-e4e2f6f9a2bf tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:20.799380 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5270f7f9-374f-485b-b800-e4e2f6f9a2bf tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:20.799795 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5270f7f9-374f-485b-b800-e4e2f6f9a2bf tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:20.931750 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-5270f7f9-374f-485b-b800-e4e2f6f9a2bf tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:20.941325 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-5270f7f9-374f-485b-b800-e4e2f6f9a2bf tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:20.943991 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-5270f7f9-374f-485b-b800-e4e2f6f9a2bf tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:20.944331 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-5270f7f9-374f-485b-b800-e4e2f6f9a2bf tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1743 microversion: 2.20 time: 0.158661 Sep 04 01:11:20.944843 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 471/965] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:20 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1743 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:11:21.727768 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b4a08a04-f583-4b41-8cf3-00656e7b4055 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:21.728381 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-b4a08a04-f583-4b41-8cf3-00656e7b4055 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 2718d3b6-c71c-4ae1-9544-0f8e4a43d989] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:21.735517 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b4a08a04-f583-4b41-8cf3-00656e7b4055 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:21.735999 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b4a08a04-f583-4b41-8cf3-00656e7b4055 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:21.736439 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b4a08a04-f583-4b41-8cf3-00656e7b4055 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:21.737345 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b4a08a04-f583-4b41-8cf3-00656e7b4055 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:21.737827 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b4a08a04-f583-4b41-8cf3-00656e7b4055 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:21.738257 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b4a08a04-f583-4b41-8cf3-00656e7b4055 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:21.918604 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-b4a08a04-f583-4b41-8cf3-00656e7b4055 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:21.927976 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-b4a08a04-f583-4b41-8cf3-00656e7b4055 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989" status: 200 len: 1366 microversion: 2.1 time: 0.204054 Sep 04 01:11:21.929124 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 495/966] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:11:21 2022] GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989 => generated 1366 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:11:21.970848 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-8c6204fb-54ef-4689-8be0-ad13973c8495 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:21.971434 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-8c6204fb-54ef-4689-8be0-ad13973c8495 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:21.983941 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8c6204fb-54ef-4689-8be0-ad13973c8495 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:21.984866 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8c6204fb-54ef-4689-8be0-ad13973c8495 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:21.988121 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8c6204fb-54ef-4689-8be0-ad13973c8495 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:21.990648 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8c6204fb-54ef-4689-8be0-ad13973c8495 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:21.992011 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8c6204fb-54ef-4689-8be0-ad13973c8495 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:21.994002 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8c6204fb-54ef-4689-8be0-ad13973c8495 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:22.029739 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:11:22.161847 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-8c6204fb-54ef-4689-8be0-ad13973c8495 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:22.172389 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-8c6204fb-54ef-4689-8be0-ad13973c8495 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:22.173986 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-8c6204fb-54ef-4689-8be0-ad13973c8495 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:22.175193 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-8c6204fb-54ef-4689-8be0-ad13973c8495 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1738 microversion: 2.20 time: 0.208930 Sep 04 01:11:22.175748 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 472/967] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:21 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1738 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:11:22.952264 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-2fcfd36f-27be-4d95-bb77-c88872413779 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:22.953676 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-2fcfd36f-27be-4d95-bb77-c88872413779 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 2718d3b6-c71c-4ae1-9544-0f8e4a43d989] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:22.960605 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2fcfd36f-27be-4d95-bb77-c88872413779 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:22.963849 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2fcfd36f-27be-4d95-bb77-c88872413779 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:22.964361 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2fcfd36f-27be-4d95-bb77-c88872413779 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:22.965319 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2fcfd36f-27be-4d95-bb77-c88872413779 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:22.968633 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2fcfd36f-27be-4d95-bb77-c88872413779 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:22.978471 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2fcfd36f-27be-4d95-bb77-c88872413779 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:23.152462 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-2fcfd36f-27be-4d95-bb77-c88872413779 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:23.164645 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-2fcfd36f-27be-4d95-bb77-c88872413779 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989" status: 200 len: 1366 microversion: 2.1 time: 0.216905 Sep 04 01:11:23.165293 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 496/968] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:11:22 2022] GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989 => generated 1366 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:11:23.204765 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-62e1fea0-a34a-4e7e-ad4a-bebe11025f02 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:23.205541 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-62e1fea0-a34a-4e7e-ad4a-bebe11025f02 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:23.213448 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell0:SELECT=2 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:11:23.214387 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell0:SELECT=8 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:11:23.217726 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-62e1fea0-a34a-4e7e-ad4a-bebe11025f02 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:23.218384 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-62e1fea0-a34a-4e7e-ad4a-bebe11025f02 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:23.221227 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-62e1fea0-a34a-4e7e-ad4a-bebe11025f02 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:23.221227 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-62e1fea0-a34a-4e7e-ad4a-bebe11025f02 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:23.221643 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-62e1fea0-a34a-4e7e-ad4a-bebe11025f02 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:23.222757 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-62e1fea0-a34a-4e7e-ad4a-bebe11025f02 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:23.392459 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-62e1fea0-a34a-4e7e-ad4a-bebe11025f02 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:23.401843 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-62e1fea0-a34a-4e7e-ad4a-bebe11025f02 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:23.403724 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-62e1fea0-a34a-4e7e-ad4a-bebe11025f02 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:23.406017 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-62e1fea0-a34a-4e7e-ad4a-bebe11025f02 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1738 microversion: 2.20 time: 0.206825 Sep 04 01:11:23.406586 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 473/969] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:23 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1738 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:11:23.962745 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-5ca826b8-9f7d-4861-bd66-c091a5d12651 req-222734e8-667d-4ba6-876b-de0989acf944 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2718d3b6-c71c-4ae1-9544-0f8e4a43d989", "name": "network-vif-plugged", "status": "completed", "tag": "a684da0f-7a2f-4974-8aa3-30875caa5a0e"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:11:23.984640 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-5ca826b8-9f7d-4861-bd66-c091a5d12651 req-222734e8-667d-4ba6-876b-de0989acf944 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:23.985417 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-5ca826b8-9f7d-4861-bd66-c091a5d12651 req-222734e8-667d-4ba6-876b-de0989acf944 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:23.986010 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-5ca826b8-9f7d-4861-bd66-c091a5d12651 req-222734e8-667d-4ba6-876b-de0989acf944 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:23.999426 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-5ca826b8-9f7d-4861-bd66-c091a5d12651 req-222734e8-667d-4ba6-876b-de0989acf944 service nova] Creating event network-vif-plugged:a684da0f-7a2f-4974-8aa3-30875caa5a0e for instance 2718d3b6-c71c-4ae1-9544-0f8e4a43d989 on hv-cinder-maste Sep 04 01:11:24.005796 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-5ca826b8-9f7d-4861-bd66-c091a5d12651 req-222734e8-667d-4ba6-876b-de0989acf944 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.214950 Sep 04 01:11:24.006867 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 497/970] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:11:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:11:24.190518 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-29cff37b-1118-4263-8e48-246fdda28b77 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:24.191615 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-29cff37b-1118-4263-8e48-246fdda28b77 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 2718d3b6-c71c-4ae1-9544-0f8e4a43d989] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:24.198667 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-29cff37b-1118-4263-8e48-246fdda28b77 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:24.199527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-29cff37b-1118-4263-8e48-246fdda28b77 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:24.200195 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-29cff37b-1118-4263-8e48-246fdda28b77 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:24.201259 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-29cff37b-1118-4263-8e48-246fdda28b77 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:24.202545 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-29cff37b-1118-4263-8e48-246fdda28b77 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:24.203013 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-29cff37b-1118-4263-8e48-246fdda28b77 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:24.386176 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-29cff37b-1118-4263-8e48-246fdda28b77 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:24.398396 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-29cff37b-1118-4263-8e48-246fdda28b77 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989" status: 200 len: 1366 microversion: 2.1 time: 0.212044 Sep 04 01:11:24.400611 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 474/971] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:11:24 2022] GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989 => generated 1366 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:11:24.438249 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-87d5c133-e59a-4d3d-9bc1-eae7d46f56c4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:24.438946 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-87d5c133-e59a-4d3d-9bc1-eae7d46f56c4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:24.445975 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-87d5c133-e59a-4d3d-9bc1-eae7d46f56c4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:24.446856 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-87d5c133-e59a-4d3d-9bc1-eae7d46f56c4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:24.447566 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-87d5c133-e59a-4d3d-9bc1-eae7d46f56c4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:24.448356 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-87d5c133-e59a-4d3d-9bc1-eae7d46f56c4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:24.448873 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-87d5c133-e59a-4d3d-9bc1-eae7d46f56c4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:24.449548 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-87d5c133-e59a-4d3d-9bc1-eae7d46f56c4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:24.652827 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-87d5c133-e59a-4d3d-9bc1-eae7d46f56c4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:24.666932 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-87d5c133-e59a-4d3d-9bc1-eae7d46f56c4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:24.668470 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-87d5c133-e59a-4d3d-9bc1-eae7d46f56c4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:24.672295 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-87d5c133-e59a-4d3d-9bc1-eae7d46f56c4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1738 microversion: 2.20 time: 0.236510 Sep 04 01:11:24.686005 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 498/972] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:24 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1738 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:11:25.423576 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-e99668a7-83d5-4563-83d5-e7f7cabf3311 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:25.424151 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-e99668a7-83d5-4563-83d5-e7f7cabf3311 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 2718d3b6-c71c-4ae1-9544-0f8e4a43d989] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:25.430953 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e99668a7-83d5-4563-83d5-e7f7cabf3311 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:25.431444 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e99668a7-83d5-4563-83d5-e7f7cabf3311 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:25.431902 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e99668a7-83d5-4563-83d5-e7f7cabf3311 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:25.432568 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e99668a7-83d5-4563-83d5-e7f7cabf3311 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:25.433191 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e99668a7-83d5-4563-83d5-e7f7cabf3311 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:25.433617 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-e99668a7-83d5-4563-83d5-e7f7cabf3311 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:25.614229 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-e99668a7-83d5-4563-83d5-e7f7cabf3311 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:25.624758 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-e99668a7-83d5-4563-83d5-e7f7cabf3311 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989" status: 200 len: 1366 microversion: 2.1 time: 0.205775 Sep 04 01:11:25.625397 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 475/973] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:11:25 2022] GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989 => generated 1366 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:11:25.695859 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-5029c469-6321-48f8-b326-73a3bd5a86c4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:25.696476 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-5029c469-6321-48f8-b326-73a3bd5a86c4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:25.704488 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5029c469-6321-48f8-b326-73a3bd5a86c4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:25.705129 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5029c469-6321-48f8-b326-73a3bd5a86c4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:25.706234 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5029c469-6321-48f8-b326-73a3bd5a86c4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:25.707227 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5029c469-6321-48f8-b326-73a3bd5a86c4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:25.707873 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5029c469-6321-48f8-b326-73a3bd5a86c4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:25.708311 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5029c469-6321-48f8-b326-73a3bd5a86c4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:25.917542 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-5029c469-6321-48f8-b326-73a3bd5a86c4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:25.928920 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-5029c469-6321-48f8-b326-73a3bd5a86c4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:25.930546 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-5029c469-6321-48f8-b326-73a3bd5a86c4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:25.931987 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-5029c469-6321-48f8-b326-73a3bd5a86c4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1738 microversion: 2.20 time: 0.240164 Sep 04 01:11:25.932641 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 499/974] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:25 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1738 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:11:26.022287 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-efdedc10-33a1-4bdb-867c-c5a2f319c6d1 req-7b8c6ce9-f929-4780-84f3-dc52de8d9248 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "da1d8cad-5ea0-48fc-a5af-81eec3d1e94b", "name": "network-vif-plugged", "status": "completed", "tag": "19f203f1-14db-4bb0-aaad-c8e57c75d249"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:11:26.036853 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-efdedc10-33a1-4bdb-867c-c5a2f319c6d1 req-7b8c6ce9-f929-4780-84f3-dc52de8d9248 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:26.037574 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-efdedc10-33a1-4bdb-867c-c5a2f319c6d1 req-7b8c6ce9-f929-4780-84f3-dc52de8d9248 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:26.038719 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-efdedc10-33a1-4bdb-867c-c5a2f319c6d1 req-7b8c6ce9-f929-4780-84f3-dc52de8d9248 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:26.055815 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-efdedc10-33a1-4bdb-867c-c5a2f319c6d1 req-7b8c6ce9-f929-4780-84f3-dc52de8d9248 service nova] Creating event network-vif-plugged:19f203f1-14db-4bb0-aaad-c8e57c75d249 for instance da1d8cad-5ea0-48fc-a5af-81eec3d1e94b on hv-cinder-maste Sep 04 01:11:26.062472 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-efdedc10-33a1-4bdb-867c-c5a2f319c6d1 req-7b8c6ce9-f929-4780-84f3-dc52de8d9248 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043829 Sep 04 01:11:26.064007 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 476/975] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:11:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:11:26.647666 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-2f3ab81c-7c0e-4f6f-9bc7-83ad90dca2b2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:26.650639 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-2f3ab81c-7c0e-4f6f-9bc7-83ad90dca2b2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 2718d3b6-c71c-4ae1-9544-0f8e4a43d989] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:26.658417 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2f3ab81c-7c0e-4f6f-9bc7-83ad90dca2b2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:26.658903 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2f3ab81c-7c0e-4f6f-9bc7-83ad90dca2b2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:26.659336 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2f3ab81c-7c0e-4f6f-9bc7-83ad90dca2b2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:26.660067 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2f3ab81c-7c0e-4f6f-9bc7-83ad90dca2b2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:26.660514 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2f3ab81c-7c0e-4f6f-9bc7-83ad90dca2b2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:26.660933 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2f3ab81c-7c0e-4f6f-9bc7-83ad90dca2b2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:26.848929 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-2f3ab81c-7c0e-4f6f-9bc7-83ad90dca2b2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:26.857419 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-2f3ab81c-7c0e-4f6f-9bc7-83ad90dca2b2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989" status: 200 len: 1557 microversion: 2.1 time: 0.213904 Sep 04 01:11:26.858057 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 500/976] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:11:26 2022] GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989 => generated 1557 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:11:26.886495 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-2c309ae6-59d5-49d9-b6ed-684343b1dbf4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:26.887801 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-2c309ae6-59d5-49d9-b6ed-684343b1dbf4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 2718d3b6-c71c-4ae1-9544-0f8e4a43d989] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:26.896050 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2c309ae6-59d5-49d9-b6ed-684343b1dbf4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:26.896953 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2c309ae6-59d5-49d9-b6ed-684343b1dbf4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:26.898177 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2c309ae6-59d5-49d9-b6ed-684343b1dbf4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:26.898355 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2c309ae6-59d5-49d9-b6ed-684343b1dbf4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:26.898971 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2c309ae6-59d5-49d9-b6ed-684343b1dbf4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:26.899559 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2c309ae6-59d5-49d9-b6ed-684343b1dbf4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:26.959601 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-24c0ab55-a320-47dc-bb55-2eae3d7aa078 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:26.960577 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-24c0ab55-a320-47dc-bb55-2eae3d7aa078 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:26.973630 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-24c0ab55-a320-47dc-bb55-2eae3d7aa078 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:26.975537 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-24c0ab55-a320-47dc-bb55-2eae3d7aa078 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:26.976168 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-24c0ab55-a320-47dc-bb55-2eae3d7aa078 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:26.977151 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-24c0ab55-a320-47dc-bb55-2eae3d7aa078 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:26.980734 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-24c0ab55-a320-47dc-bb55-2eae3d7aa078 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:26.981469 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-24c0ab55-a320-47dc-bb55-2eae3d7aa078 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:27.027602 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-2c309ae6-59d5-49d9-b6ed-684343b1dbf4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.145763 Sep 04 01:11:27.028193 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 477/977] 10.222.0.42 () {58 vars in 1275 bytes} [Sun Sep 4 01:11:26 2022] GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989/os-interface => generated 285 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:11:27.177551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-24c0ab55-a320-47dc-bb55-2eae3d7aa078 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:27.187518 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-24c0ab55-a320-47dc-bb55-2eae3d7aa078 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:27.190979 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-24c0ab55-a320-47dc-bb55-2eae3d7aa078 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:27.193668 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-24c0ab55-a320-47dc-bb55-2eae3d7aa078 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1738 microversion: 2.20 time: 0.238725 Sep 04 01:11:27.194631 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 501/978] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:26 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1738 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:11:28.224723 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-c23f3df9-e32e-451d-9b04-6f1284406207 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:28.225461 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-c23f3df9-e32e-451d-9b04-6f1284406207 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:28.244295 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c23f3df9-e32e-451d-9b04-6f1284406207 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:28.244787 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c23f3df9-e32e-451d-9b04-6f1284406207 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:28.256951 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c23f3df9-e32e-451d-9b04-6f1284406207 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:28.258011 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c23f3df9-e32e-451d-9b04-6f1284406207 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:28.258492 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c23f3df9-e32e-451d-9b04-6f1284406207 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:28.258923 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c23f3df9-e32e-451d-9b04-6f1284406207 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:28.409453 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-ed7cd4b8-1a50-41f8-9d3c-7787447ddb3d req-6e9845c8-61ff-4153-bd1c-d2e2d7059bab service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2718d3b6-c71c-4ae1-9544-0f8e4a43d989", "tag": "a684da0f-7a2f-4974-8aa3-30875caa5a0e"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:11:28.437273 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-ed7cd4b8-1a50-41f8-9d3c-7787447ddb3d req-6e9845c8-61ff-4153-bd1c-d2e2d7059bab service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:28.437273 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-ed7cd4b8-1a50-41f8-9d3c-7787447ddb3d req-6e9845c8-61ff-4153-bd1c-d2e2d7059bab service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:28.437273 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-ed7cd4b8-1a50-41f8-9d3c-7787447ddb3d req-6e9845c8-61ff-4153-bd1c-d2e2d7059bab service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:28.445775 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-ed7cd4b8-1a50-41f8-9d3c-7787447ddb3d req-6e9845c8-61ff-4153-bd1c-d2e2d7059bab service nova] Creating event network-changed:a684da0f-7a2f-4974-8aa3-30875caa5a0e for instance 2718d3b6-c71c-4ae1-9544-0f8e4a43d989 on hv-cinder-maste Sep 04 01:11:28.475854 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-ed7cd4b8-1a50-41f8-9d3c-7787447ddb3d req-6e9845c8-61ff-4153-bd1c-d2e2d7059bab service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.070528 Sep 04 01:11:28.485008 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 502/979] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:11:28 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:11:28.571827 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-c23f3df9-e32e-451d-9b04-6f1284406207 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:28.587954 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-c23f3df9-e32e-451d-9b04-6f1284406207 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:28.589533 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-c23f3df9-e32e-451d-9b04-6f1284406207 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:28.593881 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-c23f3df9-e32e-451d-9b04-6f1284406207 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1738 microversion: 2.20 time: 0.373863 Sep 04 01:11:28.607822 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 478/980] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:28 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1738 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:11:28.680873 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-8af130c6-465c-48f0-826d-0eae63aae010 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:28.684811 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-8af130c6-465c-48f0-826d-0eae63aae010 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:28.695851 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8af130c6-465c-48f0-826d-0eae63aae010 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:28.696599 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8af130c6-465c-48f0-826d-0eae63aae010 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:28.697240 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8af130c6-465c-48f0-826d-0eae63aae010 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:28.698173 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8af130c6-465c-48f0-826d-0eae63aae010 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:28.698791 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8af130c6-465c-48f0-826d-0eae63aae010 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:28.699395 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8af130c6-465c-48f0-826d-0eae63aae010 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:28.746006 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-8af130c6-465c-48f0-826d-0eae63aae010 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.070268 Sep 04 01:11:28.746757 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 503/981] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 4 01:11:28 2022] GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219/os-volume_attachments => generated 197 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:11:28.757729 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-61e1ad48-d3c2-4e86-8323-2b362e626ccb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:28.758454 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-61e1ad48-d3c2-4e86-8323-2b362e626ccb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:28.765578 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-61e1ad48-d3c2-4e86-8323-2b362e626ccb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:28.766246 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-61e1ad48-d3c2-4e86-8323-2b362e626ccb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:28.766693 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-61e1ad48-d3c2-4e86-8323-2b362e626ccb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:28.767385 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-61e1ad48-d3c2-4e86-8323-2b362e626ccb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:28.767833 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-61e1ad48-d3c2-4e86-8323-2b362e626ccb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:28.768252 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-61e1ad48-d3c2-4e86-8323-2b362e626ccb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:28.800042 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-61e1ad48-d3c2-4e86-8323-2b362e626ccb tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219/os-volume_attachments/d0664352-6a12-4f27-b283-95de87dddc60" status: 200 len: 194 microversion: 2.1 time: 0.046300 Sep 04 01:11:28.800946 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 479/982] 10.222.0.42 () {58 vars in 1413 bytes} [Sun Sep 4 01:11:28 2022] GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219/os-volume_attachments/d0664352-6a12-4f27-b283-95de87dddc60 => generated 194 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:11:29.623823 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-5261cc53-c8f2-45ef-afb7-8a0b6c27fc1e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:29.624506 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-5261cc53-c8f2-45ef-afb7-8a0b6c27fc1e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:29.631901 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5261cc53-c8f2-45ef-afb7-8a0b6c27fc1e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:29.632521 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5261cc53-c8f2-45ef-afb7-8a0b6c27fc1e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:29.633103 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5261cc53-c8f2-45ef-afb7-8a0b6c27fc1e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:29.634117 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5261cc53-c8f2-45ef-afb7-8a0b6c27fc1e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:29.634818 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5261cc53-c8f2-45ef-afb7-8a0b6c27fc1e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:29.635431 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5261cc53-c8f2-45ef-afb7-8a0b6c27fc1e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:29.834114 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-5261cc53-c8f2-45ef-afb7-8a0b6c27fc1e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:29.847020 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-5261cc53-c8f2-45ef-afb7-8a0b6c27fc1e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:29.849324 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-5261cc53-c8f2-45ef-afb7-8a0b6c27fc1e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:29.851666 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-5261cc53-c8f2-45ef-afb7-8a0b6c27fc1e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1738 microversion: 2.20 time: 0.232332 Sep 04 01:11:29.852909 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 504/983] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:29 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1738 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:11:30.663765 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-ff9e4d18-607b-4200-948b-b6f2b40fa863 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "3f2a02b0-073a-40d9-afc6-22906bd07065"}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:11:30.667007 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-ff9e4d18-607b-4200-948b-b6f2b40fa863 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:30.673894 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ff9e4d18-607b-4200-948b-b6f2b40fa863 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:30.674481 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ff9e4d18-607b-4200-948b-b6f2b40fa863 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:30.674921 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ff9e4d18-607b-4200-948b-b6f2b40fa863 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:30.678242 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ff9e4d18-607b-4200-948b-b6f2b40fa863 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:30.678242 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ff9e4d18-607b-4200-948b-b6f2b40fa863 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:30.678242 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ff9e4d18-607b-4200-948b-b6f2b40fa863 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:30.885695 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-bd82dc14-f6b1-43a6-8de4-148b8e6c53cb tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:30.887553 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-bd82dc14-f6b1-43a6-8de4-148b8e6c53cb tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:30.898350 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bd82dc14-f6b1-43a6-8de4-148b8e6c53cb tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:30.899191 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bd82dc14-f6b1-43a6-8de4-148b8e6c53cb tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:30.899978 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bd82dc14-f6b1-43a6-8de4-148b8e6c53cb tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:30.900789 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bd82dc14-f6b1-43a6-8de4-148b8e6c53cb tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:30.901275 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bd82dc14-f6b1-43a6-8de4-148b8e6c53cb tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:30.901698 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bd82dc14-f6b1-43a6-8de4-148b8e6c53cb tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:31.132676 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-bd82dc14-f6b1-43a6-8de4-148b8e6c53cb tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:31.142486 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-bd82dc14-f6b1-43a6-8de4-148b8e6c53cb tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:31.143941 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-bd82dc14-f6b1-43a6-8de4-148b8e6c53cb tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:31.145127 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-bd82dc14-f6b1-43a6-8de4-148b8e6c53cb tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1738 microversion: 2.20 time: 0.266293 Sep 04 01:11:31.146284 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 505/984] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:30 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1738 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:11:31.528522 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-ff9e4d18-607b-4200-948b-b6f2b40fa863 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "POST /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.868480 Sep 04 01:11:31.529876 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 480/985] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Sep 4 01:11:30 2022] POST /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219/os-volume_attachments => generated 194 bytes in 871 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:11:32.173086 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-8eb9c424-15d7-43c6-997a-e734bd54da71 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:32.174085 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-8eb9c424-15d7-43c6-997a-e734bd54da71 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:32.182702 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8eb9c424-15d7-43c6-997a-e734bd54da71 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:32.187283 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8eb9c424-15d7-43c6-997a-e734bd54da71 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:32.187728 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8eb9c424-15d7-43c6-997a-e734bd54da71 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:32.188454 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8eb9c424-15d7-43c6-997a-e734bd54da71 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:32.188947 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8eb9c424-15d7-43c6-997a-e734bd54da71 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:32.189424 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8eb9c424-15d7-43c6-997a-e734bd54da71 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:32.333384 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-8eb9c424-15d7-43c6-997a-e734bd54da71 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:32.341514 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-8eb9c424-15d7-43c6-997a-e734bd54da71 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:32.343094 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-8eb9c424-15d7-43c6-997a-e734bd54da71 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:32.344406 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-8eb9c424-15d7-43c6-997a-e734bd54da71 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1738 microversion: 2.20 time: 0.177441 Sep 04 01:11:32.344992 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 506/986] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:32 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1738 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:11:33.371446 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-7bb3c6a8-8d71-460a-8e32-29a00ec52227 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:33.372164 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-7bb3c6a8-8d71-460a-8e32-29a00ec52227 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:33.380273 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7bb3c6a8-8d71-460a-8e32-29a00ec52227 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:33.380839 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7bb3c6a8-8d71-460a-8e32-29a00ec52227 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:33.381224 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7bb3c6a8-8d71-460a-8e32-29a00ec52227 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:33.381948 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7bb3c6a8-8d71-460a-8e32-29a00ec52227 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:33.382418 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7bb3c6a8-8d71-460a-8e32-29a00ec52227 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:33.383090 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7bb3c6a8-8d71-460a-8e32-29a00ec52227 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:33.516420 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-7bb3c6a8-8d71-460a-8e32-29a00ec52227 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:33.524427 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-7bb3c6a8-8d71-460a-8e32-29a00ec52227 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:33.525906 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-7bb3c6a8-8d71-460a-8e32-29a00ec52227 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:33.527084 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-7bb3c6a8-8d71-460a-8e32-29a00ec52227 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1738 microversion: 2.20 time: 0.160650 Sep 04 01:11:33.527642 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 481/987] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:33 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1738 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:11:34.553396 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-e48c6d6e-45ae-42b6-89fb-8d360053ce36 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:34.554002 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-e48c6d6e-45ae-42b6-89fb-8d360053ce36 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:34.561354 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e48c6d6e-45ae-42b6-89fb-8d360053ce36 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:34.561840 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e48c6d6e-45ae-42b6-89fb-8d360053ce36 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:34.562283 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e48c6d6e-45ae-42b6-89fb-8d360053ce36 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:34.562977 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e48c6d6e-45ae-42b6-89fb-8d360053ce36 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:34.563431 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e48c6d6e-45ae-42b6-89fb-8d360053ce36 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:34.563855 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e48c6d6e-45ae-42b6-89fb-8d360053ce36 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:34.702333 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e48c6d6e-45ae-42b6-89fb-8d360053ce36 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:34.710378 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e48c6d6e-45ae-42b6-89fb-8d360053ce36 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:34.711870 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e48c6d6e-45ae-42b6-89fb-8d360053ce36 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:34.713032 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-e48c6d6e-45ae-42b6-89fb-8d360053ce36 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1720 microversion: 2.20 time: 0.163897 Sep 04 01:11:34.713645 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 507/988] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:34 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1720 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:11:35.746826 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-5ee75d88-2a47-40d5-b0e1-90520f63ab0a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:35.747433 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-5ee75d88-2a47-40d5-b0e1-90520f63ab0a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:35.754684 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5ee75d88-2a47-40d5-b0e1-90520f63ab0a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:35.755164 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5ee75d88-2a47-40d5-b0e1-90520f63ab0a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:35.755598 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5ee75d88-2a47-40d5-b0e1-90520f63ab0a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:35.756279 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5ee75d88-2a47-40d5-b0e1-90520f63ab0a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:35.756728 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5ee75d88-2a47-40d5-b0e1-90520f63ab0a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:35.757175 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5ee75d88-2a47-40d5-b0e1-90520f63ab0a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:36.161624 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-5ee75d88-2a47-40d5-b0e1-90520f63ab0a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:36.170209 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-5ee75d88-2a47-40d5-b0e1-90520f63ab0a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:36.171649 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-5ee75d88-2a47-40d5-b0e1-90520f63ab0a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:36.172794 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-5ee75d88-2a47-40d5-b0e1-90520f63ab0a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1720 microversion: 2.20 time: 0.429888 Sep 04 01:11:36.173563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 482/989] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:35 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1720 bytes in 431 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:11:37.198535 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-a23b1833-017e-4fa8-9918-0b35cd67c5b4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:37.199207 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-a23b1833-017e-4fa8-9918-0b35cd67c5b4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:37.205949 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a23b1833-017e-4fa8-9918-0b35cd67c5b4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:37.206426 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a23b1833-017e-4fa8-9918-0b35cd67c5b4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:37.206861 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a23b1833-017e-4fa8-9918-0b35cd67c5b4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:37.207535 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a23b1833-017e-4fa8-9918-0b35cd67c5b4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:37.207984 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a23b1833-017e-4fa8-9918-0b35cd67c5b4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:37.208404 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a23b1833-017e-4fa8-9918-0b35cd67c5b4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:37.341382 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-a23b1833-017e-4fa8-9918-0b35cd67c5b4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:37.350369 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-a23b1833-017e-4fa8-9918-0b35cd67c5b4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:37.351799 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-a23b1833-017e-4fa8-9918-0b35cd67c5b4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:37.352953 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-a23b1833-017e-4fa8-9918-0b35cd67c5b4 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1723 microversion: 2.20 time: 0.158325 Sep 04 01:11:37.353531 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 508/990] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:37 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1723 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:11:37.380235 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-025bc240-cf5d-49a3-a4e9-7db7b5fdaa4a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "e2d58890-b794-4c8f-9be2-422bf4f2d7d6"}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:11:37.382698 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-025bc240-cf5d-49a3-a4e9-7db7b5fdaa4a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:37.389277 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-025bc240-cf5d-49a3-a4e9-7db7b5fdaa4a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:37.389756 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-025bc240-cf5d-49a3-a4e9-7db7b5fdaa4a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:37.390178 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-025bc240-cf5d-49a3-a4e9-7db7b5fdaa4a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:37.390854 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-025bc240-cf5d-49a3-a4e9-7db7b5fdaa4a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:37.391297 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-025bc240-cf5d-49a3-a4e9-7db7b5fdaa4a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:37.391714 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-025bc240-cf5d-49a3-a4e9-7db7b5fdaa4a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:37.731698 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-1bbc2298-9943-47f6-9023-90464183ea5d req-15f44fb4-96f5-4f8e-a628-e47f0f6f983c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "da1d8cad-5ea0-48fc-a5af-81eec3d1e94b", "name": "network-vif-unplugged", "status": "completed", "tag": "19f203f1-14db-4bb0-aaad-c8e57c75d249"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:11:37.742293 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-1bbc2298-9943-47f6-9023-90464183ea5d req-15f44fb4-96f5-4f8e-a628-e47f0f6f983c service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:37.742773 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-1bbc2298-9943-47f6-9023-90464183ea5d req-15f44fb4-96f5-4f8e-a628-e47f0f6f983c service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:37.743195 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-1bbc2298-9943-47f6-9023-90464183ea5d req-15f44fb4-96f5-4f8e-a628-e47f0f6f983c service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:37.753932 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.compute.server_external_events [req-1bbc2298-9943-47f6-9023-90464183ea5d req-15f44fb4-96f5-4f8e-a628-e47f0f6f983c service nova] Unable to find a host for instance da1d8cad-5ea0-48fc-a5af-81eec3d1e94b. Dropping event network-vif-unplugged {{(pid=121036) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:144}} Sep 04 01:11:37.755017 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-1bbc2298-9943-47f6-9023-90464183ea5d req-15f44fb4-96f5-4f8e-a628-e47f0f6f983c service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.027035 Sep 04 01:11:37.755411 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 509/991] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:11:37 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 28 msecs (HTTP/1.1 207) 9 headers in 367 bytes (2 switches on core 0) Sep 04 01:11:38.340320 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-025bc240-cf5d-49a3-a4e9-7db7b5fdaa4a tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "POST /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.964033 Sep 04 01:11:38.340941 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 483/992] 10.222.0.42 () {62 vars in 1364 bytes} [Sun Sep 4 01:11:37 2022] POST /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b/os-volume_attachments => generated 188 bytes in 965 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 04 01:11:38.470088 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b1b89f11-ed1c-4326-98ca-c43c1ccb578f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:38.470611 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-b1b89f11-ed1c-4326-98ca-c43c1ccb578f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:38.477019 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b1b89f11-ed1c-4326-98ca-c43c1ccb578f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:38.477565 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b1b89f11-ed1c-4326-98ca-c43c1ccb578f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:38.478009 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b1b89f11-ed1c-4326-98ca-c43c1ccb578f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:38.478698 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b1b89f11-ed1c-4326-98ca-c43c1ccb578f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:38.479183 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b1b89f11-ed1c-4326-98ca-c43c1ccb578f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:38.479609 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b1b89f11-ed1c-4326-98ca-c43c1ccb578f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:38.999260 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=2,nova_cell1:UPDATE=2 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:11:39.078018 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-b1b89f11-ed1c-4326-98ca-c43c1ccb578f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "DELETE /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b/os-volume_attachments/e2d58890-b794-4c8f-9be2-422bf4f2d7d6" status: 202 len: 0 microversion: 2.20 time: 0.612367 Sep 04 01:11:39.078662 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 510/993] 10.222.0.42 () {60 vars in 1457 bytes} [Sun Sep 4 01:11:38 2022] DELETE /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b/os-volume_attachments/e2d58890-b794-4c8f-9be2-422bf4f2d7d6 => generated 0 bytes in 614 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 04 01:11:39.167998 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-0c1a4eef-10e2-418f-9f5f-a9619d24b2cf tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:11:39.168570 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-0c1a4eef-10e2-418f-9f5f-a9619d24b2cf tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:39.176062 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0c1a4eef-10e2-418f-9f5f-a9619d24b2cf tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:39.176537 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0c1a4eef-10e2-418f-9f5f-a9619d24b2cf tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:39.176966 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0c1a4eef-10e2-418f-9f5f-a9619d24b2cf tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:39.177678 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0c1a4eef-10e2-418f-9f5f-a9619d24b2cf tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:39.178122 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0c1a4eef-10e2-418f-9f5f-a9619d24b2cf tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:39.178660 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0c1a4eef-10e2-418f-9f5f-a9619d24b2cf tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:39.226640 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-0c1a4eef-10e2-418f-9f5f-a9619d24b2cf tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=121037) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Sep 04 01:11:39.451625 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-0c1a4eef-10e2-418f-9f5f-a9619d24b2cf tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "POST /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b/action" status: 202 len: 0 microversion: 2.20 time: 0.287536 Sep 04 01:11:39.452235 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 484/994] 10.222.0.42 () {62 vars in 1319 bytes} [Sun Sep 4 01:11:39 2022] POST /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b/action => generated 0 bytes in 289 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 04 01:11:39.460649 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-64c6d774-83d6-49f9-8b3b-650bc7ee71c6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:39.461298 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-64c6d774-83d6-49f9-8b3b-650bc7ee71c6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:39.468302 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-64c6d774-83d6-49f9-8b3b-650bc7ee71c6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:39.468787 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-64c6d774-83d6-49f9-8b3b-650bc7ee71c6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:39.469253 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-64c6d774-83d6-49f9-8b3b-650bc7ee71c6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:39.470070 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-64c6d774-83d6-49f9-8b3b-650bc7ee71c6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:39.470525 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-64c6d774-83d6-49f9-8b3b-650bc7ee71c6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:39.471071 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-64c6d774-83d6-49f9-8b3b-650bc7ee71c6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:39.613515 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-64c6d774-83d6-49f9-8b3b-650bc7ee71c6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:39.623289 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-64c6d774-83d6-49f9-8b3b-650bc7ee71c6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:39.624778 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-64c6d774-83d6-49f9-8b3b-650bc7ee71c6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:39.625996 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-64c6d774-83d6-49f9-8b3b-650bc7ee71c6 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1671 microversion: 2.20 time: 0.169533 Sep 04 01:11:39.626549 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 511/995] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:39 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1671 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:11:40.213903 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-9709ac3a-95f8-4940-8979-72db1464df00 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:40.216171 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-9709ac3a-95f8-4940-8979-72db1464df00 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:40.223834 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9709ac3a-95f8-4940-8979-72db1464df00 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:40.224331 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9709ac3a-95f8-4940-8979-72db1464df00 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:40.224990 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9709ac3a-95f8-4940-8979-72db1464df00 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:40.225888 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9709ac3a-95f8-4940-8979-72db1464df00 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:40.226194 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9709ac3a-95f8-4940-8979-72db1464df00 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:40.226623 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9709ac3a-95f8-4940-8979-72db1464df00 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:40.242821 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=88,nova_cell1:UPDATE=7,nova_cell1:INSERT=5 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:11:40.261087 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-9709ac3a-95f8-4940-8979-72db1464df00 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.051798 Sep 04 01:11:40.261662 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 485/996] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 4 01:11:40 2022] GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219/os-volume_attachments => generated 371 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:11:40.415041 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-1a0cf90a-d0f5-4c35-97f3-a1fca7efab80 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:40.416035 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-1a0cf90a-d0f5-4c35-97f3-a1fca7efab80 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:40.433942 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1a0cf90a-d0f5-4c35-97f3-a1fca7efab80 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:40.434442 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1a0cf90a-d0f5-4c35-97f3-a1fca7efab80 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:40.434884 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1a0cf90a-d0f5-4c35-97f3-a1fca7efab80 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:40.435720 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1a0cf90a-d0f5-4c35-97f3-a1fca7efab80 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:40.436036 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1a0cf90a-d0f5-4c35-97f3-a1fca7efab80 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:40.436463 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1a0cf90a-d0f5-4c35-97f3-a1fca7efab80 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:40.476513 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-1a0cf90a-d0f5-4c35-97f3-a1fca7efab80 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219/os-volume_attachments/d0664352-6a12-4f27-b283-95de87dddc60" status: 200 len: 194 microversion: 2.1 time: 0.066786 Sep 04 01:11:40.477266 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 512/997] 10.222.0.42 () {58 vars in 1413 bytes} [Sun Sep 4 01:11:40 2022] GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219/os-volume_attachments/d0664352-6a12-4f27-b283-95de87dddc60 => generated 194 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:11:40.488580 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-482a3938-3297-405a-800e-929f89b7dd09 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:40.489197 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-482a3938-3297-405a-800e-929f89b7dd09 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:40.495846 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-482a3938-3297-405a-800e-929f89b7dd09 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:40.496341 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-482a3938-3297-405a-800e-929f89b7dd09 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:40.496832 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-482a3938-3297-405a-800e-929f89b7dd09 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:40.497629 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-482a3938-3297-405a-800e-929f89b7dd09 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:40.498220 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-482a3938-3297-405a-800e-929f89b7dd09 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:40.498631 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-482a3938-3297-405a-800e-929f89b7dd09 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:40.589663 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.objects.instance [None req-482a3938-3297-405a-800e-929f89b7dd09 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lazy-loading 'services' on Instance uuid 52505c46-cb03-49ec-a7ba-696d7c6ac219 {{(pid=121037) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:11:40.652076 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-19dd63d4-b1cb-4601-8671-0c5d6c2d8967 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:40.652707 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-19dd63d4-b1cb-4601-8671-0c5d6c2d8967 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:40.659635 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-19dd63d4-b1cb-4601-8671-0c5d6c2d8967 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:40.660110 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-19dd63d4-b1cb-4601-8671-0c5d6c2d8967 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:40.660540 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-19dd63d4-b1cb-4601-8671-0c5d6c2d8967 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:40.661350 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-19dd63d4-b1cb-4601-8671-0c5d6c2d8967 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:40.661802 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-19dd63d4-b1cb-4601-8671-0c5d6c2d8967 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:40.662219 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-19dd63d4-b1cb-4601-8671-0c5d6c2d8967 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:40.800220 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-19dd63d4-b1cb-4601-8671-0c5d6c2d8967 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:40.808572 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-19dd63d4-b1cb-4601-8671-0c5d6c2d8967 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:40.810215 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-19dd63d4-b1cb-4601-8671-0c5d6c2d8967 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:40.811652 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-19dd63d4-b1cb-4601-8671-0c5d6c2d8967 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1671 microversion: 2.20 time: 0.163812 Sep 04 01:11:40.812283 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 513/998] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:40 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1671 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:11:40.849877 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-482a3938-3297-405a-800e-929f89b7dd09 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "DELETE /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219/os-volume_attachments/d0664352-6a12-4f27-b283-95de87dddc60" status: 202 len: 0 microversion: 2.1 time: 0.365352 Sep 04 01:11:40.850639 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 486/999] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Sep 4 01:11:40 2022] DELETE /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219/os-volume_attachments/d0664352-6a12-4f27-b283-95de87dddc60 => generated 0 bytes in 367 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 04 01:11:41.837885 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-fd790744-c825-4579-8523-48d150904a5d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:41.838618 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-fd790744-c825-4579-8523-48d150904a5d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:41.845519 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fd790744-c825-4579-8523-48d150904a5d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:41.846097 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fd790744-c825-4579-8523-48d150904a5d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:41.846530 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fd790744-c825-4579-8523-48d150904a5d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:41.847368 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fd790744-c825-4579-8523-48d150904a5d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:41.847894 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fd790744-c825-4579-8523-48d150904a5d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:41.848320 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fd790744-c825-4579-8523-48d150904a5d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:41.995950 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-fd790744-c825-4579-8523-48d150904a5d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:42.006071 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-fd790744-c825-4579-8523-48d150904a5d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:42.007558 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-fd790744-c825-4579-8523-48d150904a5d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:42.008720 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-fd790744-c825-4579-8523-48d150904a5d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1729 microversion: 2.20 time: 0.175180 Sep 04 01:11:42.009398 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 514/1000] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:41 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1729 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:11:43.036614 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-3a41e1f4-6a1f-49c4-a643-6f6773f5fc3d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:43.037371 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-3a41e1f4-6a1f-49c4-a643-6f6773f5fc3d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:43.043698 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3a41e1f4-6a1f-49c4-a643-6f6773f5fc3d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:43.044180 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3a41e1f4-6a1f-49c4-a643-6f6773f5fc3d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:43.044727 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3a41e1f4-6a1f-49c4-a643-6f6773f5fc3d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:43.045459 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3a41e1f4-6a1f-49c4-a643-6f6773f5fc3d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:43.045913 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3a41e1f4-6a1f-49c4-a643-6f6773f5fc3d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:43.046338 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-3a41e1f4-6a1f-49c4-a643-6f6773f5fc3d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:43.046462 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-0b1e12a9-8d20-4a29-b8ce-e7f7579ffade tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:43.046985 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-0b1e12a9-8d20-4a29-b8ce-e7f7579ffade tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:43.053377 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0b1e12a9-8d20-4a29-b8ce-e7f7579ffade tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:43.053855 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0b1e12a9-8d20-4a29-b8ce-e7f7579ffade tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:43.054282 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0b1e12a9-8d20-4a29-b8ce-e7f7579ffade tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:43.054968 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0b1e12a9-8d20-4a29-b8ce-e7f7579ffade tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:43.055438 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0b1e12a9-8d20-4a29-b8ce-e7f7579ffade tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:43.055856 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0b1e12a9-8d20-4a29-b8ce-e7f7579ffade tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:43.085780 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-0b1e12a9-8d20-4a29-b8ce-e7f7579ffade tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219/os-volume_attachments/3f2a02b0-073a-40d9-afc6-22906bd07065" status: 200 len: 194 microversion: 2.1 time: 0.043591 Sep 04 01:11:43.086271 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 515/1001] 10.222.0.42 () {58 vars in 1413 bytes} [Sun Sep 4 01:11:43 2022] GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219/os-volume_attachments/3f2a02b0-073a-40d9-afc6-22906bd07065 => generated 194 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:11:43.097672 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-864ad6d5-86e2-4775-be02-5d75ac626ea4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:43.098161 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-864ad6d5-86e2-4775-be02-5d75ac626ea4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:43.104014 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-864ad6d5-86e2-4775-be02-5d75ac626ea4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:43.104490 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-864ad6d5-86e2-4775-be02-5d75ac626ea4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:43.104921 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-864ad6d5-86e2-4775-be02-5d75ac626ea4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:43.105627 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-864ad6d5-86e2-4775-be02-5d75ac626ea4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:43.106075 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-864ad6d5-86e2-4775-be02-5d75ac626ea4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:43.106495 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-864ad6d5-86e2-4775-be02-5d75ac626ea4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:43.181790 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-3a41e1f4-6a1f-49c4-a643-6f6773f5fc3d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:43.191612 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-3a41e1f4-6a1f-49c4-a643-6f6773f5fc3d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:43.193218 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-3a41e1f4-6a1f-49c4-a643-6f6773f5fc3d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:43.194636 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-3a41e1f4-6a1f-49c4-a643-6f6773f5fc3d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1729 microversion: 2.20 time: 0.163046 Sep 04 01:11:43.195495 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 487/1002] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:43 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1729 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:11:43.196257 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.objects.instance [None req-864ad6d5-86e2-4775-be02-5d75ac626ea4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lazy-loading 'services' on Instance uuid 52505c46-cb03-49ec-a7ba-696d7c6ac219 {{(pid=121036) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:11:43.460405 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-864ad6d5-86e2-4775-be02-5d75ac626ea4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "DELETE /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219/os-volume_attachments/3f2a02b0-073a-40d9-afc6-22906bd07065" status: 202 len: 0 microversion: 2.1 time: 0.366658 Sep 04 01:11:43.461141 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 516/1003] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Sep 4 01:11:43 2022] DELETE /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219/os-volume_attachments/3f2a02b0-073a-40d9-afc6-22906bd07065 => generated 0 bytes in 368 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 04 01:11:44.221156 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-6f675e9b-3e2c-4efb-ba42-e138c4c7e255 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:44.222473 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-6f675e9b-3e2c-4efb-ba42-e138c4c7e255 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:44.230036 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6f675e9b-3e2c-4efb-ba42-e138c4c7e255 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:44.230780 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6f675e9b-3e2c-4efb-ba42-e138c4c7e255 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:44.231379 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6f675e9b-3e2c-4efb-ba42-e138c4c7e255 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:44.233223 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6f675e9b-3e2c-4efb-ba42-e138c4c7e255 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:44.233675 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6f675e9b-3e2c-4efb-ba42-e138c4c7e255 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:44.234377 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6f675e9b-3e2c-4efb-ba42-e138c4c7e255 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:44.367749 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-6f675e9b-3e2c-4efb-ba42-e138c4c7e255 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:44.379220 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-6f675e9b-3e2c-4efb-ba42-e138c4c7e255 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:44.381311 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-6f675e9b-3e2c-4efb-ba42-e138c4c7e255 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:44.383131 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-6f675e9b-3e2c-4efb-ba42-e138c4c7e255 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1729 microversion: 2.20 time: 0.166654 Sep 04 01:11:44.386016 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 488/1004] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:44 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1729 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:11:45.411704 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-0e5d1a46-2d9c-4cda-8bcf-e7a97babae82 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:45.412308 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-0e5d1a46-2d9c-4cda-8bcf-e7a97babae82 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:45.414734 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_api:SELECT=60 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:11:45.419613 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0e5d1a46-2d9c-4cda-8bcf-e7a97babae82 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:45.420101 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0e5d1a46-2d9c-4cda-8bcf-e7a97babae82 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:45.420520 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0e5d1a46-2d9c-4cda-8bcf-e7a97babae82 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:45.421238 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0e5d1a46-2d9c-4cda-8bcf-e7a97babae82 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:45.421687 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0e5d1a46-2d9c-4cda-8bcf-e7a97babae82 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:45.422104 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0e5d1a46-2d9c-4cda-8bcf-e7a97babae82 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:45.559303 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-0e5d1a46-2d9c-4cda-8bcf-e7a97babae82 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:45.569935 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-0e5d1a46-2d9c-4cda-8bcf-e7a97babae82 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:45.571664 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-0e5d1a46-2d9c-4cda-8bcf-e7a97babae82 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:45.572970 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-0e5d1a46-2d9c-4cda-8bcf-e7a97babae82 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1729 microversion: 2.20 time: 0.165622 Sep 04 01:11:45.574082 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 517/1005] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:45 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1729 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:11:46.604106 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-2637979b-d7d0-49ae-bb66-ada8e1a3c5a2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:46.604868 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-2637979b-d7d0-49ae-bb66-ada8e1a3c5a2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:46.612188 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2637979b-d7d0-49ae-bb66-ada8e1a3c5a2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:46.612749 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2637979b-d7d0-49ae-bb66-ada8e1a3c5a2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:46.613302 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2637979b-d7d0-49ae-bb66-ada8e1a3c5a2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:46.614070 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2637979b-d7d0-49ae-bb66-ada8e1a3c5a2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:46.614599 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2637979b-d7d0-49ae-bb66-ada8e1a3c5a2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:46.615105 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2637979b-d7d0-49ae-bb66-ada8e1a3c5a2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:46.765681 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-2637979b-d7d0-49ae-bb66-ada8e1a3c5a2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:46.780784 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-2637979b-d7d0-49ae-bb66-ada8e1a3c5a2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:46.782720 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-2637979b-d7d0-49ae-bb66-ada8e1a3c5a2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:46.784271 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-2637979b-d7d0-49ae-bb66-ada8e1a3c5a2 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1729 microversion: 2.20 time: 0.184351 Sep 04 01:11:46.785322 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 489/1006] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:46 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1729 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:11:47.810536 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-9234dab9-271b-4a7f-93c9-df1a633dba7c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:47.811748 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-9234dab9-271b-4a7f-93c9-df1a633dba7c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:47.818396 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9234dab9-271b-4a7f-93c9-df1a633dba7c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:47.819103 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9234dab9-271b-4a7f-93c9-df1a633dba7c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:47.819709 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9234dab9-271b-4a7f-93c9-df1a633dba7c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:47.820489 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9234dab9-271b-4a7f-93c9-df1a633dba7c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:47.820941 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9234dab9-271b-4a7f-93c9-df1a633dba7c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:47.821597 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9234dab9-271b-4a7f-93c9-df1a633dba7c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:47.984182 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-9234dab9-271b-4a7f-93c9-df1a633dba7c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:47.992677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-9234dab9-271b-4a7f-93c9-df1a633dba7c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:47.994241 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-9234dab9-271b-4a7f-93c9-df1a633dba7c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:47.995399 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-9234dab9-271b-4a7f-93c9-df1a633dba7c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1729 microversion: 2.20 time: 0.188698 Sep 04 01:11:47.995959 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 518/1007] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:47 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1729 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:11:49.168997 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-ea579feb-f33f-4579-a3c0-61905811ddb8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:49.168997 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-ea579feb-f33f-4579-a3c0-61905811ddb8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:49.168997 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea579feb-f33f-4579-a3c0-61905811ddb8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:49.168997 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea579feb-f33f-4579-a3c0-61905811ddb8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:49.170592 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea579feb-f33f-4579-a3c0-61905811ddb8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:49.170592 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea579feb-f33f-4579-a3c0-61905811ddb8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:49.170592 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea579feb-f33f-4579-a3c0-61905811ddb8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:49.170592 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea579feb-f33f-4579-a3c0-61905811ddb8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:49.193401 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-ea579feb-f33f-4579-a3c0-61905811ddb8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:49.206685 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-ea579feb-f33f-4579-a3c0-61905811ddb8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:49.208752 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-ea579feb-f33f-4579-a3c0-61905811ddb8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:49.210257 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-ea579feb-f33f-4579-a3c0-61905811ddb8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1729 microversion: 2.20 time: 0.194185 Sep 04 01:11:49.223805 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 490/1008] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:49 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1729 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:11:50.240462 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-cdc209e7-75fd-428a-95c8-a9c4c850a7fe tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:50.241185 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-cdc209e7-75fd-428a-95c8-a9c4c850a7fe tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:50.250101 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cdc209e7-75fd-428a-95c8-a9c4c850a7fe tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:50.253377 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cdc209e7-75fd-428a-95c8-a9c4c850a7fe tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:50.253377 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cdc209e7-75fd-428a-95c8-a9c4c850a7fe tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:50.253377 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cdc209e7-75fd-428a-95c8-a9c4c850a7fe tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:50.253377 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cdc209e7-75fd-428a-95c8-a9c4c850a7fe tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:50.253976 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cdc209e7-75fd-428a-95c8-a9c4c850a7fe tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:50.432840 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-cdc209e7-75fd-428a-95c8-a9c4c850a7fe tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:50.442245 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-cdc209e7-75fd-428a-95c8-a9c4c850a7fe tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:50.443964 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-cdc209e7-75fd-428a-95c8-a9c4c850a7fe tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:50.445362 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-cdc209e7-75fd-428a-95c8-a9c4c850a7fe tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1729 microversion: 2.20 time: 0.209048 Sep 04 01:11:50.446431 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 519/1009] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:50 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1729 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:11:51.474274 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-cae26297-d06d-4211-8cb5-9b0862236751 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:51.476124 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-cae26297-d06d-4211-8cb5-9b0862236751 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:51.477412 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_api:SELECT=46,nova_api:UPDATE=1 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:11:51.482094 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-cae26297-d06d-4211-8cb5-9b0862236751 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:51.482641 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-cae26297-d06d-4211-8cb5-9b0862236751 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:51.483102 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-cae26297-d06d-4211-8cb5-9b0862236751 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:51.483895 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-cae26297-d06d-4211-8cb5-9b0862236751 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:51.484352 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-cae26297-d06d-4211-8cb5-9b0862236751 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:51.484774 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-cae26297-d06d-4211-8cb5-9b0862236751 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:51.659777 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-cae26297-d06d-4211-8cb5-9b0862236751 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:51.676932 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-cae26297-d06d-4211-8cb5-9b0862236751 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:51.679208 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-cae26297-d06d-4211-8cb5-9b0862236751 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:51.682040 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-cae26297-d06d-4211-8cb5-9b0862236751 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1729 microversion: 2.20 time: 0.211236 Sep 04 01:11:51.682813 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 491/1010] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:51 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1729 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:11:52.710877 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-893769c8-2f80-464c-bb7c-253d51ef1ee7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:52.711464 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-893769c8-2f80-464c-bb7c-253d51ef1ee7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:52.718755 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-893769c8-2f80-464c-bb7c-253d51ef1ee7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:52.719224 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-893769c8-2f80-464c-bb7c-253d51ef1ee7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:52.719650 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-893769c8-2f80-464c-bb7c-253d51ef1ee7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:52.720335 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-893769c8-2f80-464c-bb7c-253d51ef1ee7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:52.720781 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-893769c8-2f80-464c-bb7c-253d51ef1ee7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:52.721233 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-893769c8-2f80-464c-bb7c-253d51ef1ee7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:52.902412 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-893769c8-2f80-464c-bb7c-253d51ef1ee7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:52.913134 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-893769c8-2f80-464c-bb7c-253d51ef1ee7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:52.913429 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-893769c8-2f80-464c-bb7c-253d51ef1ee7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:52.914332 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-893769c8-2f80-464c-bb7c-253d51ef1ee7 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1729 microversion: 2.20 time: 0.207729 Sep 04 01:11:52.914884 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 520/1011] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:52 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1729 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:11:53.290485 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-b9ff7939-f1ce-4e59-8d15-9426a64d968e tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:11:53.292092 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-b9ff7939-f1ce-4e59-8d15-9426a64d968e tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:53.303130 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b9ff7939-f1ce-4e59-8d15-9426a64d968e tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:53.303621 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b9ff7939-f1ce-4e59-8d15-9426a64d968e tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:53.304081 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b9ff7939-f1ce-4e59-8d15-9426a64d968e tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:53.304940 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b9ff7939-f1ce-4e59-8d15-9426a64d968e tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:53.305447 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b9ff7939-f1ce-4e59-8d15-9426a64d968e tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:53.306143 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b9ff7939-f1ce-4e59-8d15-9426a64d968e tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:53.333126 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.objects.instance [None req-b9ff7939-f1ce-4e59-8d15-9426a64d968e tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lazy-loading 'flavor' on Instance uuid 74df11dc-31bd-4745-83bc-4b3c430d02ff {{(pid=121037) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:11:53.940777 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-4ebfd19f-0a6f-4552-bd3b-99ecb200ecb9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:53.941667 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-4ebfd19f-0a6f-4552-bd3b-99ecb200ecb9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:53.958497 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4ebfd19f-0a6f-4552-bd3b-99ecb200ecb9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:53.960788 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4ebfd19f-0a6f-4552-bd3b-99ecb200ecb9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:53.961502 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4ebfd19f-0a6f-4552-bd3b-99ecb200ecb9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:53.963446 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4ebfd19f-0a6f-4552-bd3b-99ecb200ecb9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:53.964859 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4ebfd19f-0a6f-4552-bd3b-99ecb200ecb9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:53.969540 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4ebfd19f-0a6f-4552-bd3b-99ecb200ecb9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:54.044332 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-b9ff7939-f1ce-4e59-8d15-9426a64d968e tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "POST /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff/action" status: 202 len: 0 microversion: 2.1 time: 0.758147 Sep 04 01:11:54.044332 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 492/1012] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Sep 4 01:11:53 2022] POST /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff/action => generated 0 bytes in 760 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 04 01:11:54.054456 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-6b2276eb-f3da-4c31-a345-6d2a24412663 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:54.055165 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-6b2276eb-f3da-4c31-a345-6d2a24412663 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:54.072882 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6b2276eb-f3da-4c31-a345-6d2a24412663 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:54.073499 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6b2276eb-f3da-4c31-a345-6d2a24412663 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:54.073966 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6b2276eb-f3da-4c31-a345-6d2a24412663 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:54.074664 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6b2276eb-f3da-4c31-a345-6d2a24412663 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:54.075111 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6b2276eb-f3da-4c31-a345-6d2a24412663 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:54.075553 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6b2276eb-f3da-4c31-a345-6d2a24412663 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:54.115855 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-4ebfd19f-0a6f-4552-bd3b-99ecb200ecb9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:54.124205 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-4ebfd19f-0a6f-4552-bd3b-99ecb200ecb9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:54.125657 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-4ebfd19f-0a6f-4552-bd3b-99ecb200ecb9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:54.126804 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-4ebfd19f-0a6f-4552-bd3b-99ecb200ecb9 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1729 microversion: 2.20 time: 0.191539 Sep 04 01:11:54.127662 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 521/1013] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:53 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1729 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:11:54.236956 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-6b2276eb-f3da-4c31-a345-6d2a24412663 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:54.254601 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-6b2276eb-f3da-4c31-a345-6d2a24412663 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1568 microversion: 2.1 time: 0.204554 Sep 04 01:11:54.255807 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 493/1014] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:11:54 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1568 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:11:55.152885 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-03166aba-a6a6-4f75-950e-6e910b35d565 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:55.153511 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-03166aba-a6a6-4f75-950e-6e910b35d565 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:55.160118 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-03166aba-a6a6-4f75-950e-6e910b35d565 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:55.160600 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-03166aba-a6a6-4f75-950e-6e910b35d565 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:55.161141 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-03166aba-a6a6-4f75-950e-6e910b35d565 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:55.161836 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-03166aba-a6a6-4f75-950e-6e910b35d565 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:55.162290 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-03166aba-a6a6-4f75-950e-6e910b35d565 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:55.162715 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-03166aba-a6a6-4f75-950e-6e910b35d565 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:55.280068 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-85726d5f-f39f-4855-906a-e34251b9efab tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:55.280755 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-85726d5f-f39f-4855-906a-e34251b9efab tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:55.287813 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-85726d5f-f39f-4855-906a-e34251b9efab tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:55.288302 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-85726d5f-f39f-4855-906a-e34251b9efab tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:55.288737 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-85726d5f-f39f-4855-906a-e34251b9efab tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:55.289496 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-85726d5f-f39f-4855-906a-e34251b9efab tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:55.289940 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-85726d5f-f39f-4855-906a-e34251b9efab tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:55.290356 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-85726d5f-f39f-4855-906a-e34251b9efab tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:55.293403 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-03166aba-a6a6-4f75-950e-6e910b35d565 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:55.302906 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-03166aba-a6a6-4f75-950e-6e910b35d565 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:55.304837 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-03166aba-a6a6-4f75-950e-6e910b35d565 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:55.305792 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-03166aba-a6a6-4f75-950e-6e910b35d565 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1729 microversion: 2.20 time: 0.156994 Sep 04 01:11:55.306595 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 522/1015] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:55 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1729 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:11:55.424010 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-85726d5f-f39f-4855-906a-e34251b9efab tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:55.431844 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-85726d5f-f39f-4855-906a-e34251b9efab tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1568 microversion: 2.1 time: 0.156029 Sep 04 01:11:55.432508 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 494/1016] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:11:55 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1568 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:11:56.331678 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b72dfdff-8ff6-4233-80ad-ac737d39cfaf tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:56.332280 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-b72dfdff-8ff6-4233-80ad-ac737d39cfaf tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:56.338953 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b72dfdff-8ff6-4233-80ad-ac737d39cfaf tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:56.339423 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b72dfdff-8ff6-4233-80ad-ac737d39cfaf tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:56.339887 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b72dfdff-8ff6-4233-80ad-ac737d39cfaf tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:56.340563 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b72dfdff-8ff6-4233-80ad-ac737d39cfaf tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:56.341018 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b72dfdff-8ff6-4233-80ad-ac737d39cfaf tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:56.341504 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b72dfdff-8ff6-4233-80ad-ac737d39cfaf tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:56.456706 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-34d9751f-3120-4238-b19c-bb2ee1981236 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:56.457429 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-34d9751f-3120-4238-b19c-bb2ee1981236 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:56.469284 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-34d9751f-3120-4238-b19c-bb2ee1981236 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:56.469284 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-34d9751f-3120-4238-b19c-bb2ee1981236 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:56.469284 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-34d9751f-3120-4238-b19c-bb2ee1981236 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:56.469284 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-34d9751f-3120-4238-b19c-bb2ee1981236 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:56.470078 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-34d9751f-3120-4238-b19c-bb2ee1981236 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:56.470078 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-34d9751f-3120-4238-b19c-bb2ee1981236 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:56.474688 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-b72dfdff-8ff6-4233-80ad-ac737d39cfaf tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:56.527854 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-b72dfdff-8ff6-4233-80ad-ac737d39cfaf tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:56.527854 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-b72dfdff-8ff6-4233-80ad-ac737d39cfaf tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:56.527854 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-b72dfdff-8ff6-4233-80ad-ac737d39cfaf tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1729 microversion: 2.20 time: 0.160623 Sep 04 01:11:56.528763 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 523/1017] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:56 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1729 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:11:56.613060 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-34d9751f-3120-4238-b19c-bb2ee1981236 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:56.621862 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-34d9751f-3120-4238-b19c-bb2ee1981236 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1568 microversion: 2.1 time: 0.169126 Sep 04 01:11:56.622430 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 495/1018] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:11:56 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1568 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:11:57.514006 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-dc9c4578-7f3d-4cc0-82ff-570f9c050a27 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:57.514612 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-dc9c4578-7f3d-4cc0-82ff-570f9c050a27 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:57.521030 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dc9c4578-7f3d-4cc0-82ff-570f9c050a27 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:57.521575 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dc9c4578-7f3d-4cc0-82ff-570f9c050a27 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:57.522005 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dc9c4578-7f3d-4cc0-82ff-570f9c050a27 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:57.522699 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dc9c4578-7f3d-4cc0-82ff-570f9c050a27 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:57.523150 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dc9c4578-7f3d-4cc0-82ff-570f9c050a27 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:57.523574 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dc9c4578-7f3d-4cc0-82ff-570f9c050a27 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:57.646075 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-15e78686-6a01-4eec-9dbf-5cf0901deb96 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:57.646691 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-15e78686-6a01-4eec-9dbf-5cf0901deb96 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:57.653378 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-15e78686-6a01-4eec-9dbf-5cf0901deb96 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:57.653872 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-15e78686-6a01-4eec-9dbf-5cf0901deb96 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:57.654358 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-15e78686-6a01-4eec-9dbf-5cf0901deb96 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:57.655297 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-15e78686-6a01-4eec-9dbf-5cf0901deb96 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:57.655756 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-15e78686-6a01-4eec-9dbf-5cf0901deb96 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:57.656184 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-15e78686-6a01-4eec-9dbf-5cf0901deb96 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:57.666547 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-dc9c4578-7f3d-4cc0-82ff-570f9c050a27 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:57.674924 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-dc9c4578-7f3d-4cc0-82ff-570f9c050a27 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:57.676373 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-dc9c4578-7f3d-4cc0-82ff-570f9c050a27 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:57.677732 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-dc9c4578-7f3d-4cc0-82ff-570f9c050a27 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1729 microversion: 2.20 time: 0.167618 Sep 04 01:11:57.678295 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 524/1019] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:57 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1729 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:11:57.838354 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-15e78686-6a01-4eec-9dbf-5cf0901deb96 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:57.847261 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-15e78686-6a01-4eec-9dbf-5cf0901deb96 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1568 microversion: 2.1 time: 0.204952 Sep 04 01:11:57.857288 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 496/1020] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:11:57 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1568 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:11:58.516547 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-8ddd9167-2d29-436b-8b93-6ae4b9e106a0 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:58.518978 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-8ddd9167-2d29-436b-8b93-6ae4b9e106a0 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:58.525375 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8ddd9167-2d29-436b-8b93-6ae4b9e106a0 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:58.526965 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8ddd9167-2d29-436b-8b93-6ae4b9e106a0 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:58.526965 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8ddd9167-2d29-436b-8b93-6ae4b9e106a0 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:58.526965 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8ddd9167-2d29-436b-8b93-6ae4b9e106a0 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:58.527383 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8ddd9167-2d29-436b-8b93-6ae4b9e106a0 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:58.527802 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8ddd9167-2d29-436b-8b93-6ae4b9e106a0 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:58.557846 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-8ddd9167-2d29-436b-8b93-6ae4b9e106a0 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.045853 Sep 04 01:11:58.559535 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 525/1021] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 4 01:11:58 2022] GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219/os-volume_attachments => generated 25 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 04 01:11:58.689615 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-0f63743a-f952-4162-945d-1e8052efa5ba tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:58.689615 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-0f63743a-f952-4162-945d-1e8052efa5ba tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:58.689615 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0f63743a-f952-4162-945d-1e8052efa5ba tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:58.689615 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0f63743a-f952-4162-945d-1e8052efa5ba tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:58.690398 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0f63743a-f952-4162-945d-1e8052efa5ba tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:58.690398 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0f63743a-f952-4162-945d-1e8052efa5ba tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:58.690398 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0f63743a-f952-4162-945d-1e8052efa5ba tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:58.690398 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0f63743a-f952-4162-945d-1e8052efa5ba tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:58.705323 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-8de9ea55-b722-4d92-8957-b4b64d0c5c75 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:58.705982 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-8de9ea55-b722-4d92-8957-b4b64d0c5c75 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:58.711936 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8de9ea55-b722-4d92-8957-b4b64d0c5c75 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:58.712411 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8de9ea55-b722-4d92-8957-b4b64d0c5c75 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:58.712843 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8de9ea55-b722-4d92-8957-b4b64d0c5c75 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:58.713555 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8de9ea55-b722-4d92-8957-b4b64d0c5c75 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:58.714003 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8de9ea55-b722-4d92-8957-b4b64d0c5c75 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:58.714419 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8de9ea55-b722-4d92-8957-b4b64d0c5c75 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:58.718489 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:11:58.719021 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-0f63743a-f952-4162-945d-1e8052efa5ba tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.045167 Sep 04 01:11:58.719448 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 497/1022] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 4 01:11:58 2022] GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219/os-volume_attachments => generated 25 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 04 01:11:58.788929 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-83cef295-edb9-47e1-b646-5908732faae5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:58.789457 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-83cef295-edb9-47e1-b646-5908732faae5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:58.795982 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-83cef295-edb9-47e1-b646-5908732faae5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:58.796472 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-83cef295-edb9-47e1-b646-5908732faae5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:58.796915 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-83cef295-edb9-47e1-b646-5908732faae5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:58.797645 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-83cef295-edb9-47e1-b646-5908732faae5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:58.798101 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-83cef295-edb9-47e1-b646-5908732faae5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:58.798528 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-83cef295-edb9-47e1-b646-5908732faae5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:58.831389 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-83cef295-edb9-47e1-b646-5908732faae5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Going to try to terminate instance {{(pid=121037) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 04 01:11:58.846738 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=2,nova_cell1:UPDATE=2 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:11:58.859405 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-8de9ea55-b722-4d92-8957-b4b64d0c5c75 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:58.867918 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-8de9ea55-b722-4d92-8957-b4b64d0c5c75 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:58.869692 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-8de9ea55-b722-4d92-8957-b4b64d0c5c75 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:58.870861 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-8de9ea55-b722-4d92-8957-b4b64d0c5c75 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1729 microversion: 2.20 time: 0.171003 Sep 04 01:11:58.871416 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 526/1023] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:58 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1729 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 04 01:11:58.876104 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-d47f88c9-676e-4ae3-8355-d169155ebf43 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:58.876709 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-d47f88c9-676e-4ae3-8355-d169155ebf43 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:58.882645 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d47f88c9-676e-4ae3-8355-d169155ebf43 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:58.893589 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d47f88c9-676e-4ae3-8355-d169155ebf43 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:58.893589 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d47f88c9-676e-4ae3-8355-d169155ebf43 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:58.893589 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d47f88c9-676e-4ae3-8355-d169155ebf43 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:58.893589 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d47f88c9-676e-4ae3-8355-d169155ebf43 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:58.894063 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d47f88c9-676e-4ae3-8355-d169155ebf43 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:59.026203 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-d47f88c9-676e-4ae3-8355-d169155ebf43 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:59.034903 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-d47f88c9-676e-4ae3-8355-d169155ebf43 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1568 microversion: 2.1 time: 0.162245 Sep 04 01:11:59.035467 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 527/1024] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:11:58 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1568 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:11:59.066991 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-83cef295-edb9-47e1-b646-5908732faae5 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "DELETE /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219" status: 204 len: 0 microversion: 2.1 time: 0.281582 Sep 04 01:11:59.067576 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 498/1025] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 4 01:11:58 2022] DELETE /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 04 01:11:59.075934 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-17081855-0a35-48a8-bb14-dffa0b35ecd9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:59.077204 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-17081855-0a35-48a8-bb14-dffa0b35ecd9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:59.083225 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-17081855-0a35-48a8-bb14-dffa0b35ecd9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:59.083729 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-17081855-0a35-48a8-bb14-dffa0b35ecd9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:59.084160 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-17081855-0a35-48a8-bb14-dffa0b35ecd9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:59.084842 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-17081855-0a35-48a8-bb14-dffa0b35ecd9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:59.085315 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-17081855-0a35-48a8-bb14-dffa0b35ecd9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:59.094093 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-17081855-0a35-48a8-bb14-dffa0b35ecd9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:59.222123 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-17081855-0a35-48a8-bb14-dffa0b35ecd9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:11:59.230132 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-17081855-0a35-48a8-bb14-dffa0b35ecd9 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219" status: 200 len: 1672 microversion: 2.1 time: 0.158136 Sep 04 01:11:59.230689 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 528/1026] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:11:59 2022] GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219 => generated 1672 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:11:59.898199 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-ba9fcca2-14eb-4afa-851a-c5f42b1df7fc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:11:59.898919 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-ba9fcca2-14eb-4afa-851a-c5f42b1df7fc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:11:59.906161 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ba9fcca2-14eb-4afa-851a-c5f42b1df7fc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:59.906768 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ba9fcca2-14eb-4afa-851a-c5f42b1df7fc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:59.907208 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ba9fcca2-14eb-4afa-851a-c5f42b1df7fc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:11:59.907930 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ba9fcca2-14eb-4afa-851a-c5f42b1df7fc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:11:59.908376 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ba9fcca2-14eb-4afa-851a-c5f42b1df7fc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:11:59.908797 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ba9fcca2-14eb-4afa-851a-c5f42b1df7fc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:00.059355 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-08fcc047-5dd6-4849-ac8d-00195a41c845 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:00.059951 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-08fcc047-5dd6-4849-ac8d-00195a41c845 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:00.060089 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-ba9fcca2-14eb-4afa-851a-c5f42b1df7fc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:00.067203 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-08fcc047-5dd6-4849-ac8d-00195a41c845 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:00.068575 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-08fcc047-5dd6-4849-ac8d-00195a41c845 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:00.069229 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-08fcc047-5dd6-4849-ac8d-00195a41c845 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:00.069663 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-ba9fcca2-14eb-4afa-851a-c5f42b1df7fc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:00.070189 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-08fcc047-5dd6-4849-ac8d-00195a41c845 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:00.070640 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-08fcc047-5dd6-4849-ac8d-00195a41c845 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:00.071065 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-08fcc047-5dd6-4849-ac8d-00195a41c845 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:00.071176 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-ba9fcca2-14eb-4afa-851a-c5f42b1df7fc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:00.072334 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-ba9fcca2-14eb-4afa-851a-c5f42b1df7fc tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1729 microversion: 2.20 time: 0.178545 Sep 04 01:12:00.085047 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 499/1027] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:11:59 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1729 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:12:00.209333 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-08fcc047-5dd6-4849-ac8d-00195a41c845 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:00.217633 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-08fcc047-5dd6-4849-ac8d-00195a41c845 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1568 microversion: 2.1 time: 0.161560 Sep 04 01:12:00.218269 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 529/1028] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:00 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1568 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:12:00.257196 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-6f70b6f3-b1ad-4920-b5c8-5c8423c5a1b7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:00.257806 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-6f70b6f3-b1ad-4920-b5c8-5c8423c5a1b7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:00.265139 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6f70b6f3-b1ad-4920-b5c8-5c8423c5a1b7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:00.265594 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6f70b6f3-b1ad-4920-b5c8-5c8423c5a1b7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:00.266035 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6f70b6f3-b1ad-4920-b5c8-5c8423c5a1b7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:00.266743 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6f70b6f3-b1ad-4920-b5c8-5c8423c5a1b7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:00.267198 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6f70b6f3-b1ad-4920-b5c8-5c8423c5a1b7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:00.267623 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6f70b6f3-b1ad-4920-b5c8-5c8423c5a1b7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:00.414660 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-6f70b6f3-b1ad-4920-b5c8-5c8423c5a1b7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:00.423163 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-6f70b6f3-b1ad-4920-b5c8-5c8423c5a1b7 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219" status: 200 len: 1672 microversion: 2.1 time: 0.169691 Sep 04 01:12:00.424124 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 500/1029] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:00 2022] GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219 => generated 1672 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:12:01.099877 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-674ddb5f-9ffe-4115-abea-8af5b78b0a2d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:01.100481 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-674ddb5f-9ffe-4115-abea-8af5b78b0a2d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:01.106944 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-674ddb5f-9ffe-4115-abea-8af5b78b0a2d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:01.107568 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-674ddb5f-9ffe-4115-abea-8af5b78b0a2d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:01.108016 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-674ddb5f-9ffe-4115-abea-8af5b78b0a2d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:01.108696 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-674ddb5f-9ffe-4115-abea-8af5b78b0a2d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:01.109167 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-674ddb5f-9ffe-4115-abea-8af5b78b0a2d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:01.109588 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-674ddb5f-9ffe-4115-abea-8af5b78b0a2d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:01.244184 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-674ddb5f-9ffe-4115-abea-8af5b78b0a2d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:01.248955 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-50e03f4a-6d92-43f6-b5cb-5abc940c06f1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:01.249601 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-50e03f4a-6d92-43f6-b5cb-5abc940c06f1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:01.253680 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-674ddb5f-9ffe-4115-abea-8af5b78b0a2d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:01.255380 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-674ddb5f-9ffe-4115-abea-8af5b78b0a2d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:01.255959 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-50e03f4a-6d92-43f6-b5cb-5abc940c06f1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:01.256441 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-50e03f4a-6d92-43f6-b5cb-5abc940c06f1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:01.256560 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-674ddb5f-9ffe-4115-abea-8af5b78b0a2d tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1729 microversion: 2.20 time: 0.160622 Sep 04 01:12:01.256883 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-50e03f4a-6d92-43f6-b5cb-5abc940c06f1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:01.257647 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-50e03f4a-6d92-43f6-b5cb-5abc940c06f1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:01.258260 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-50e03f4a-6d92-43f6-b5cb-5abc940c06f1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:01.258685 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-50e03f4a-6d92-43f6-b5cb-5abc940c06f1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:01.269508 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 530/1030] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:12:01 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1729 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:12:01.448074 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-06834c24-631c-4ff1-add3-34a46add42db tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:01.448625 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-06834c24-631c-4ff1-add3-34a46add42db tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:01.454893 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-06834c24-631c-4ff1-add3-34a46add42db tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:01.458368 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-06834c24-631c-4ff1-add3-34a46add42db tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:01.458368 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-06834c24-631c-4ff1-add3-34a46add42db tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:01.458368 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-06834c24-631c-4ff1-add3-34a46add42db tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:01.458368 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-06834c24-631c-4ff1-add3-34a46add42db tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:01.458740 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-06834c24-631c-4ff1-add3-34a46add42db tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:01.489188 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-50e03f4a-6d92-43f6-b5cb-5abc940c06f1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:01.498372 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-50e03f4a-6d92-43f6-b5cb-5abc940c06f1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1568 microversion: 2.1 time: 0.259824 Sep 04 01:12:01.505616 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 501/1031] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:01 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1568 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:12:01.588582 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-06834c24-631c-4ff1-add3-34a46add42db tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:01.596944 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-06834c24-631c-4ff1-add3-34a46add42db tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219" status: 200 len: 1672 microversion: 2.1 time: 0.152979 Sep 04 01:12:01.597559 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 531/1032] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:01 2022] GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219 => generated 1672 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:12:01.855983 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-1bdf854c-abcc-49ea-bc54-0c8324d82d49 req-1db7f58d-187d-46a9-ac4c-e9d719ae1b70 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "da1d8cad-5ea0-48fc-a5af-81eec3d1e94b", "name": "network-vif-plugged", "status": "completed", "tag": "19f203f1-14db-4bb0-aaad-c8e57c75d249"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:12:01.866593 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-1bdf854c-abcc-49ea-bc54-0c8324d82d49 req-1db7f58d-187d-46a9-ac4c-e9d719ae1b70 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:01.867046 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-1bdf854c-abcc-49ea-bc54-0c8324d82d49 req-1db7f58d-187d-46a9-ac4c-e9d719ae1b70 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:01.867473 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-1bdf854c-abcc-49ea-bc54-0c8324d82d49 req-1db7f58d-187d-46a9-ac4c-e9d719ae1b70 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:01.879224 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-1bdf854c-abcc-49ea-bc54-0c8324d82d49 req-1db7f58d-187d-46a9-ac4c-e9d719ae1b70 service nova] Creating event network-vif-plugged:19f203f1-14db-4bb0-aaad-c8e57c75d249 for instance da1d8cad-5ea0-48fc-a5af-81eec3d1e94b on hv-cinder-maste Sep 04 01:12:01.885549 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-1bdf854c-abcc-49ea-bc54-0c8324d82d49 req-1db7f58d-187d-46a9-ac4c-e9d719ae1b70 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033246 Sep 04 01:12:01.886938 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 502/1033] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:12:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:12:02.143118 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-9df08ba3-8234-4544-954c-cb9e0ee078b2 req-62b7c6ce-573e-45c5-bf0a-99d7bae87800 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "52505c46-cb03-49ec-a7ba-696d7c6ac219", "name": "network-vif-deleted", "tag": "73f5bdf6-a4ae-435e-9050-14b4d54218f9"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:12:02.153703 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-9df08ba3-8234-4544-954c-cb9e0ee078b2 req-62b7c6ce-573e-45c5-bf0a-99d7bae87800 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:02.154891 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-9df08ba3-8234-4544-954c-cb9e0ee078b2 req-62b7c6ce-573e-45c5-bf0a-99d7bae87800 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:02.155772 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-9df08ba3-8234-4544-954c-cb9e0ee078b2 req-62b7c6ce-573e-45c5-bf0a-99d7bae87800 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:02.168346 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-9df08ba3-8234-4544-954c-cb9e0ee078b2 req-62b7c6ce-573e-45c5-bf0a-99d7bae87800 service nova] Creating event network-vif-deleted:73f5bdf6-a4ae-435e-9050-14b4d54218f9 for instance 52505c46-cb03-49ec-a7ba-696d7c6ac219 on hv-cinder-maste Sep 04 01:12:02.174454 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-9df08ba3-8234-4544-954c-cb9e0ee078b2 req-62b7c6ce-573e-45c5-bf0a-99d7bae87800 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035220 Sep 04 01:12:02.178552 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 532/1034] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:12:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:12:02.285163 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-a80b7c4d-faf9-4b23-9f5a-959c0a27e91f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:02.285867 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-a80b7c4d-faf9-4b23-9f5a-959c0a27e91f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:02.307396 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a80b7c4d-faf9-4b23-9f5a-959c0a27e91f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:02.308141 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a80b7c4d-faf9-4b23-9f5a-959c0a27e91f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:02.308815 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a80b7c4d-faf9-4b23-9f5a-959c0a27e91f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:02.309977 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a80b7c4d-faf9-4b23-9f5a-959c0a27e91f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:02.311186 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a80b7c4d-faf9-4b23-9f5a-959c0a27e91f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:02.312395 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a80b7c4d-faf9-4b23-9f5a-959c0a27e91f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:02.481117 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-a80b7c4d-faf9-4b23-9f5a-959c0a27e91f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:02.494788 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-a80b7c4d-faf9-4b23-9f5a-959c0a27e91f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:02.496453 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-a80b7c4d-faf9-4b23-9f5a-959c0a27e91f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:02.497650 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-a80b7c4d-faf9-4b23-9f5a-959c0a27e91f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1729 microversion: 2.20 time: 0.218538 Sep 04 01:12:02.498712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 503/1035] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:12:02 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1729 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:12:02.522140 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-23762226-f085-4771-a2f9-1993fd1fb700 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:02.522859 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-23762226-f085-4771-a2f9-1993fd1fb700 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:02.529157 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-23762226-f085-4771-a2f9-1993fd1fb700 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:02.532247 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-23762226-f085-4771-a2f9-1993fd1fb700 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:02.532247 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-23762226-f085-4771-a2f9-1993fd1fb700 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:02.532247 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-23762226-f085-4771-a2f9-1993fd1fb700 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:02.532247 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-23762226-f085-4771-a2f9-1993fd1fb700 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:02.532703 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-23762226-f085-4771-a2f9-1993fd1fb700 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:02.619974 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-31f4a10d-d0fe-4fdf-8f4f-ca60d1a5ebd4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:02.620584 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-31f4a10d-d0fe-4fdf-8f4f-ca60d1a5ebd4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:02.630073 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-31f4a10d-d0fe-4fdf-8f4f-ca60d1a5ebd4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:02.632895 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-31f4a10d-d0fe-4fdf-8f4f-ca60d1a5ebd4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:02.633295 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-31f4a10d-d0fe-4fdf-8f4f-ca60d1a5ebd4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:02.635363 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-31f4a10d-d0fe-4fdf-8f4f-ca60d1a5ebd4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:02.635989 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-31f4a10d-d0fe-4fdf-8f4f-ca60d1a5ebd4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:02.637250 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-31f4a10d-d0fe-4fdf-8f4f-ca60d1a5ebd4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:02.671550 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-31f4a10d-d0fe-4fdf-8f4f-ca60d1a5ebd4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:02.672214 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-31f4a10d-d0fe-4fdf-8f4f-ca60d1a5ebd4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:02.672755 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-31f4a10d-d0fe-4fdf-8f4f-ca60d1a5ebd4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:02.673341 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.objects.instance [None req-31f4a10d-d0fe-4fdf-8f4f-ca60d1a5ebd4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lazy-loading 'fault' on Instance uuid 52505c46-cb03-49ec-a7ba-696d7c6ac219 {{(pid=121037) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:12:02.674721 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-23762226-f085-4771-a2f9-1993fd1fb700 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:02.682693 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-23762226-f085-4771-a2f9-1993fd1fb700 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1568 microversion: 2.1 time: 0.165034 Sep 04 01:12:02.683667 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 533/1036] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:02 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1568 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:12:02.718216 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-31f4a10d-d0fe-4fdf-8f4f-ca60d1a5ebd4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:02.726341 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-31f4a10d-d0fe-4fdf-8f4f-ca60d1a5ebd4 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219" status: 200 len: 1327 microversion: 2.1 time: 0.110439 Sep 04 01:12:02.727295 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 504/1037] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:02 2022] GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219 => generated 1327 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:12:03.531335 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-985ee802-4ae6-4e43-b360-cc08616649f5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:03.531943 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-985ee802-4ae6-4e43-b360-cc08616649f5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:03.538484 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-985ee802-4ae6-4e43-b360-cc08616649f5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:03.539059 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-985ee802-4ae6-4e43-b360-cc08616649f5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:03.539502 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-985ee802-4ae6-4e43-b360-cc08616649f5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:03.540201 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-985ee802-4ae6-4e43-b360-cc08616649f5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:03.540654 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-985ee802-4ae6-4e43-b360-cc08616649f5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:03.541100 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-985ee802-4ae6-4e43-b360-cc08616649f5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:03.683824 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-985ee802-4ae6-4e43-b360-cc08616649f5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:03.708359 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-4d0202c0-0d80-41af-9788-eed1ed740576 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:03.709096 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-4d0202c0-0d80-41af-9788-eed1ed740576 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:03.711827 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-985ee802-4ae6-4e43-b360-cc08616649f5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:03.713563 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-985ee802-4ae6-4e43-b360-cc08616649f5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:03.714797 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-985ee802-4ae6-4e43-b360-cc08616649f5 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1729 microversion: 2.20 time: 0.188013 Sep 04 01:12:03.725417 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 534/1038] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:12:03 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1729 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:12:03.725810 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4d0202c0-0d80-41af-9788-eed1ed740576 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:03.725810 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4d0202c0-0d80-41af-9788-eed1ed740576 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:03.725810 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4d0202c0-0d80-41af-9788-eed1ed740576 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:03.725810 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4d0202c0-0d80-41af-9788-eed1ed740576 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:03.726178 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4d0202c0-0d80-41af-9788-eed1ed740576 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:03.726178 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-4d0202c0-0d80-41af-9788-eed1ed740576 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:03.752029 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-7217363d-24ef-45ad-bb87-d30f35b06bed tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:03.752650 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-7217363d-24ef-45ad-bb87-d30f35b06bed tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:03.763158 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7217363d-24ef-45ad-bb87-d30f35b06bed tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:03.764337 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7217363d-24ef-45ad-bb87-d30f35b06bed tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:03.764530 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7217363d-24ef-45ad-bb87-d30f35b06bed tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:03.771692 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7217363d-24ef-45ad-bb87-d30f35b06bed tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:03.771692 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7217363d-24ef-45ad-bb87-d30f35b06bed tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:03.771692 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7217363d-24ef-45ad-bb87-d30f35b06bed tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:03.804801 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7217363d-24ef-45ad-bb87-d30f35b06bed tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:03.805464 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7217363d-24ef-45ad-bb87-d30f35b06bed tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:03.805907 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-7217363d-24ef-45ad-bb87-d30f35b06bed tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:03.806365 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.objects.instance [None req-7217363d-24ef-45ad-bb87-d30f35b06bed tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lazy-loading 'fault' on Instance uuid 52505c46-cb03-49ec-a7ba-696d7c6ac219 {{(pid=121036) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:12:03.874146 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-7217363d-24ef-45ad-bb87-d30f35b06bed tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '44993cd93360442bb0afb28778ee83bc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'e7dca68418a14d7da9aa63ac25f202e8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:03.882142 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-7217363d-24ef-45ad-bb87-d30f35b06bed tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219" status: 200 len: 1327 microversion: 2.1 time: 0.135491 Sep 04 01:12:03.883240 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 535/1039] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:03 2022] GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219 => generated 1327 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:12:03.916713 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-4d0202c0-0d80-41af-9788-eed1ed740576 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:03.925635 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-4d0202c0-0d80-41af-9788-eed1ed740576 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1568 microversion: 2.1 time: 0.221377 Sep 04 01:12:03.937111 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 505/1040] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:03 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1568 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:12:04.744303 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-4e4a7023-ad27-4f6b-9b4b-d768ca9cfea1 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:04.745914 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-4e4a7023-ad27-4f6b-9b4b-d768ca9cfea1 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:04.752481 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4e4a7023-ad27-4f6b-9b4b-d768ca9cfea1 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:04.752964 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4e4a7023-ad27-4f6b-9b4b-d768ca9cfea1 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:04.753431 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4e4a7023-ad27-4f6b-9b4b-d768ca9cfea1 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:04.754124 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4e4a7023-ad27-4f6b-9b4b-d768ca9cfea1 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:04.754574 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4e4a7023-ad27-4f6b-9b4b-d768ca9cfea1 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:04.755026 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4e4a7023-ad27-4f6b-9b4b-d768ca9cfea1 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:04.890777 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-4e4a7023-ad27-4f6b-9b4b-d768ca9cfea1 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:04.899084 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-4e4a7023-ad27-4f6b-9b4b-d768ca9cfea1 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:04.900731 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-4e4a7023-ad27-4f6b-9b4b-d768ca9cfea1 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:04.901916 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-4e4a7023-ad27-4f6b-9b4b-d768ca9cfea1 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1729 microversion: 2.20 time: 0.161951 Sep 04 01:12:04.902454 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 536/1041] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:12:04 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1729 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:12:04.907760 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-312dde03-5b52-4511-a2d9-8337c0c065a8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:04.908367 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-312dde03-5b52-4511-a2d9-8337c0c065a8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:04.916014 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-312dde03-5b52-4511-a2d9-8337c0c065a8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:04.916536 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-312dde03-5b52-4511-a2d9-8337c0c065a8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:04.916969 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-312dde03-5b52-4511-a2d9-8337c0c065a8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:04.917779 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-312dde03-5b52-4511-a2d9-8337c0c065a8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:04.918232 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-312dde03-5b52-4511-a2d9-8337c0c065a8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:04.918661 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-312dde03-5b52-4511-a2d9-8337c0c065a8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:04.936938 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.wsgi [None req-312dde03-5b52-4511-a2d9-8337c0c065a8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] HTTP exception thrown: Instance 52505c46-cb03-49ec-a7ba-696d7c6ac219 could not be found. Sep 04 01:12:04.938713 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-312dde03-5b52-4511-a2d9-8337c0c065a8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Request method failure captured: Sep 04 01:12:04.938713 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: request: GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219 HTTP/1.1 Sep 04 01:12:04.938713 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept: application/json Sep 04 01:12:04.938713 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept-Encoding: identity Sep 04 01:12:04.938713 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Connection: close Sep 04 01:12:04.938713 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Length: 0 Sep 04 01:12:04.938713 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Type: application/json Sep 04 01:12:04.938713 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Host: 10.222.0.42 Sep 04 01:12:04.938713 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Openstack-System-Scope: None Sep 04 01:12:04.938713 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: User-Agent: python-urllib3/1.26.12 Sep 04 01:12:04.938713 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Auth-Token: gAAAAABjE_nJhkGHWb7T6kbA22g6MQpQUnzziOpw0WCnFjRF-qzwvonIMqa9XQHefbSvkVL4GGxkzosDVGSxep3ShpT_NSR1Y0CpdR-zYyq1zOcQJ4WQ7o2EXz_hogwf8tYG4DUEXAu4uYN44FQubcx8a7LGqRtbPamZnr7PotOUcFaQv_B5v7E Sep 04 01:12:04.938713 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Id: None Sep 04 01:12:04.938713 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Name: None Sep 04 01:12:04.938713 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Identity-Status: Confirmed Sep 04 01:12:04.938713 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Is-Admin-Project: True Sep 04 01:12:04.938713 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Id: default Sep 04 01:12:04.938713 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Name: Default Sep 04 01:12:04.938713 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Id: e7dca68418a14d7da9aa63ac25f202e8 Sep 04 01:12:04.938713 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Name: tempest-AttachVolumeTestJSON-1676343226 Sep 04 01:12:04.938713 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Role: reader,member Sep 04 01:12:04.938713 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Roles: reader,member Sep 04 01:12:04.940148 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/e7dca68418a14d7da9aa63ac25f202e8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_e7dca68418a14d7da9aa63ac25f202e8", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/e7dca68418a14d7da9aa63ac25f202e8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/e7dca68418a14d7da9aa63ac25f202e8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:12:04.940148 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant: tempest-AttachVolumeTestJSON-1676343226 Sep 04 01:12:04.940148 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Id: e7dca68418a14d7da9aa63ac25f202e8 Sep 04 01:12:04.940148 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1676343226 Sep 04 01:12:04.940148 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User: tempest-AttachVolumeTestJSON-1676343226-project Sep 04 01:12:04.940148 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Id: default Sep 04 01:12:04.940148 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Name: Default Sep 04 01:12:04.940148 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Id: 44993cd93360442bb0afb28778ee83bc Sep 04 01:12:04.940148 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Name: tempest-AttachVolumeTestJSON-1676343226-project Sep 04 01:12:04.940148 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: method: > Sep 04 01:12:04.940148 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: exception: Instance 52505c46-cb03-49ec-a7ba-696d7c6ac219 could not be found. Sep 04 01:12:04.940148 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: action_args: {'id': '52505c46-cb03-49ec-a7ba-696d7c6ac219'} Sep 04 01:12:04.940922 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:12:04.940922 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:04.940922 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:12:04.940922 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:12:04.940922 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:12:04.940922 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:12:04.940922 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:12:04.940922 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:12:04.940922 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:12:04.940922 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:12:04.940922 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 52505c46-cb03-49ec-a7ba-696d7c6ac219 could not be found. Sep 04 01:12:04.940922 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:04.940922 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:04.940922 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:04.940922 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:04.940922 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:04.940922 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:04.940922 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:12:04.940922 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:12:04.940922 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:12:04.942485 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:12:04.942485 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 04 01:12:04.942485 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 04 01:12:04.942485 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:12:04.942485 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:12:04.942485 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:12:04.942485 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:12:04.942485 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 52505c46-cb03-49ec-a7ba-696d7c6ac219 could not be found. Sep 04 01:12:04.942485 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:04.942485 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:04.942485 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:04.942485 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:04.942485 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:04.942485 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:04.942485 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:12:04.942485 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:12:04.942485 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 52505c46-cb03-49ec-a7ba-696d7c6ac219 could not be found. Sep 04 01:12:04.942485 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:04.943876 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-312dde03-5b52-4511-a2d9-8337c0c065a8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Returning 404 to user: Instance 52505c46-cb03-49ec-a7ba-696d7c6ac219 could not be found. {{(pid=121037) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:12:04.943876 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-312dde03-5b52-4511-a2d9-8337c0c065a8 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219" status: 404 len: 111 microversion: 2.1 time: 0.037359 Sep 04 01:12:04.943876 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 506/1042] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:04 2022] GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 04 01:12:04.950384 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b3b50164-6705-4d35-9cf6-f80ea9b4666c tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '.version_select at 0x7fc10c569160>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:04.950908 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-57133584-703a-4480-b572-d2658a5d05db tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:04.952064 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-57133584-703a-4480-b572-d2658a5d05db tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:04.958539 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-57133584-703a-4480-b572-d2658a5d05db tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:04.959378 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-57133584-703a-4480-b572-d2658a5d05db tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:04.959818 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-57133584-703a-4480-b572-d2658a5d05db tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:04.960542 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-57133584-703a-4480-b572-d2658a5d05db tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:04.960991 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-57133584-703a-4480-b572-d2658a5d05db tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:04.961439 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-57133584-703a-4480-b572-d2658a5d05db tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:05.045277 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-b3b50164-6705-4d35-9cf6-f80ea9b4666c tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1390926425" status: 202 len: 0 microversion: 2.1 time: 0.099748 Sep 04 01:12:05.046063 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 537/1043] 10.222.0.42 () {58 vars in 1221 bytes} [Sun Sep 4 01:12:04 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1390926425 => generated 0 bytes in 101 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 04 01:12:05.099743 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-57133584-703a-4480-b572-d2658a5d05db tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:05.108862 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-57133584-703a-4480-b572-d2658a5d05db tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1568 microversion: 2.1 time: 0.162193 Sep 04 01:12:05.109474 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 507/1044] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:04 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1568 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:12:05.928371 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-3a39cf69-9933-42f7-8480-cd0cc3e607c8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:05.929009 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-3a39cf69-9933-42f7-8480-cd0cc3e607c8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:05.935231 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3a39cf69-9933-42f7-8480-cd0cc3e607c8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:05.935751 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3a39cf69-9933-42f7-8480-cd0cc3e607c8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:05.936190 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3a39cf69-9933-42f7-8480-cd0cc3e607c8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:05.936883 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3a39cf69-9933-42f7-8480-cd0cc3e607c8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:05.937371 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3a39cf69-9933-42f7-8480-cd0cc3e607c8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:05.937805 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3a39cf69-9933-42f7-8480-cd0cc3e607c8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:06.072767 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-3a39cf69-9933-42f7-8480-cd0cc3e607c8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:06.082051 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-3a39cf69-9933-42f7-8480-cd0cc3e607c8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:06.083586 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-3a39cf69-9933-42f7-8480-cd0cc3e607c8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:06.084745 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-3a39cf69-9933-42f7-8480-cd0cc3e607c8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1716 microversion: 2.20 time: 0.160907 Sep 04 01:12:06.085324 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 538/1045] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:12:05 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1716 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:12:06.133427 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-5246c129-3dcc-4189-9fc3-47449fb47089 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:06.133971 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-5246c129-3dcc-4189-9fc3-47449fb47089 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:06.140911 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5246c129-3dcc-4189-9fc3-47449fb47089 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:06.141426 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5246c129-3dcc-4189-9fc3-47449fb47089 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:06.141860 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5246c129-3dcc-4189-9fc3-47449fb47089 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:06.142547 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5246c129-3dcc-4189-9fc3-47449fb47089 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:06.143013 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5246c129-3dcc-4189-9fc3-47449fb47089 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:06.143436 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5246c129-3dcc-4189-9fc3-47449fb47089 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:06.284079 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-5246c129-3dcc-4189-9fc3-47449fb47089 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:06.291765 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-5246c129-3dcc-4189-9fc3-47449fb47089 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1568 microversion: 2.1 time: 0.162397 Sep 04 01:12:06.292322 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 508/1046] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:06 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1568 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:12:07.318798 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-fedf8392-aaf7-4021-9ec9-579d5a60fdd9 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:07.320742 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-fedf8392-aaf7-4021-9ec9-579d5a60fdd9 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:07.331346 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fedf8392-aaf7-4021-9ec9-579d5a60fdd9 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:07.333554 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fedf8392-aaf7-4021-9ec9-579d5a60fdd9 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:07.334080 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fedf8392-aaf7-4021-9ec9-579d5a60fdd9 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:07.339165 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fedf8392-aaf7-4021-9ec9-579d5a60fdd9 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:07.339651 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fedf8392-aaf7-4021-9ec9-579d5a60fdd9 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:07.340086 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fedf8392-aaf7-4021-9ec9-579d5a60fdd9 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:07.489600 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-fedf8392-aaf7-4021-9ec9-579d5a60fdd9 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:07.497592 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-fedf8392-aaf7-4021-9ec9-579d5a60fdd9 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1568 microversion: 2.1 time: 0.185584 Sep 04 01:12:07.499006 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 539/1047] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:07 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1568 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:12:08.425773 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-b5247fff-cfa9-40bf-94d9-93765b03f9e6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:08.426284 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-b5247fff-cfa9-40bf-94d9-93765b03f9e6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:08.433041 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b5247fff-cfa9-40bf-94d9-93765b03f9e6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:08.433558 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b5247fff-cfa9-40bf-94d9-93765b03f9e6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:08.434000 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b5247fff-cfa9-40bf-94d9-93765b03f9e6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:08.434701 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b5247fff-cfa9-40bf-94d9-93765b03f9e6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:08.435165 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b5247fff-cfa9-40bf-94d9-93765b03f9e6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:08.435587 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b5247fff-cfa9-40bf-94d9-93765b03f9e6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:08.449307 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.wsgi [None req-b5247fff-cfa9-40bf-94d9-93765b03f9e6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] HTTP exception thrown: Instance 52505c46-cb03-49ec-a7ba-696d7c6ac219 could not be found. Sep 04 01:12:08.450859 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-b5247fff-cfa9-40bf-94d9-93765b03f9e6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Request method failure captured: Sep 04 01:12:08.450859 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: request: DELETE /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219 HTTP/1.1 Sep 04 01:12:08.450859 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept: application/json Sep 04 01:12:08.450859 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept-Encoding: identity Sep 04 01:12:08.450859 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Connection: close Sep 04 01:12:08.450859 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Length: 0 Sep 04 01:12:08.450859 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Type: application/json Sep 04 01:12:08.450859 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Host: 10.222.0.42 Sep 04 01:12:08.450859 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Openstack-System-Scope: None Sep 04 01:12:08.450859 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: User-Agent: python-urllib3/1.26.12 Sep 04 01:12:08.450859 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Auth-Token: gAAAAABjE_nJhkGHWb7T6kbA22g6MQpQUnzziOpw0WCnFjRF-qzwvonIMqa9XQHefbSvkVL4GGxkzosDVGSxep3ShpT_NSR1Y0CpdR-zYyq1zOcQJ4WQ7o2EXz_hogwf8tYG4DUEXAu4uYN44FQubcx8a7LGqRtbPamZnr7PotOUcFaQv_B5v7E Sep 04 01:12:08.450859 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Id: None Sep 04 01:12:08.450859 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Name: None Sep 04 01:12:08.450859 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Identity-Status: Confirmed Sep 04 01:12:08.450859 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Is-Admin-Project: True Sep 04 01:12:08.450859 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Id: default Sep 04 01:12:08.450859 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Name: Default Sep 04 01:12:08.450859 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Id: e7dca68418a14d7da9aa63ac25f202e8 Sep 04 01:12:08.450859 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Name: tempest-AttachVolumeTestJSON-1676343226 Sep 04 01:12:08.450859 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Role: reader,member Sep 04 01:12:08.450859 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Roles: reader,member Sep 04 01:12:08.452363 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/e7dca68418a14d7da9aa63ac25f202e8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_e7dca68418a14d7da9aa63ac25f202e8", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/e7dca68418a14d7da9aa63ac25f202e8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/e7dca68418a14d7da9aa63ac25f202e8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:12:08.452363 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant: tempest-AttachVolumeTestJSON-1676343226 Sep 04 01:12:08.452363 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Id: e7dca68418a14d7da9aa63ac25f202e8 Sep 04 01:12:08.452363 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1676343226 Sep 04 01:12:08.452363 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User: tempest-AttachVolumeTestJSON-1676343226-project Sep 04 01:12:08.452363 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Id: default Sep 04 01:12:08.452363 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Name: Default Sep 04 01:12:08.452363 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Id: 44993cd93360442bb0afb28778ee83bc Sep 04 01:12:08.452363 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Name: tempest-AttachVolumeTestJSON-1676343226-project Sep 04 01:12:08.452363 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: method: > Sep 04 01:12:08.452363 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: exception: Instance 52505c46-cb03-49ec-a7ba-696d7c6ac219 could not be found. Sep 04 01:12:08.452363 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: action_args: {'id': '52505c46-cb03-49ec-a7ba-696d7c6ac219'} Sep 04 01:12:08.453914 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:12:08.453914 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:08.453914 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:12:08.453914 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:12:08.453914 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:12:08.453914 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:12:08.453914 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:12:08.453914 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:12:08.453914 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:12:08.453914 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:12:08.453914 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 52505c46-cb03-49ec-a7ba-696d7c6ac219 could not be found. Sep 04 01:12:08.453914 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:08.453914 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:08.453914 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:08.453914 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:08.453914 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:08.453914 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:08.453914 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:12:08.453914 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:12:08.453914 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:12:08.455093 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:12:08.455093 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 04 01:12:08.455093 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 04 01:12:08.455093 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 04 01:12:08.455093 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 04 01:12:08.455093 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:12:08.455093 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:12:08.455093 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:12:08.455093 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:12:08.455093 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 52505c46-cb03-49ec-a7ba-696d7c6ac219 could not be found. Sep 04 01:12:08.455093 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:08.455093 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:08.455093 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:08.455093 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:08.455093 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:08.455093 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:08.455093 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:12:08.455093 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:12:08.455093 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 52505c46-cb03-49ec-a7ba-696d7c6ac219 could not be found. Sep 04 01:12:08.455093 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:08.456733 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-b5247fff-cfa9-40bf-94d9-93765b03f9e6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Returning 404 to user: Instance 52505c46-cb03-49ec-a7ba-696d7c6ac219 could not be found. {{(pid=121037) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:12:08.456733 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-b5247fff-cfa9-40bf-94d9-93765b03f9e6 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "DELETE /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219" status: 404 len: 111 microversion: 2.1 time: 0.031204 Sep 04 01:12:08.456733 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 509/1048] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 4 01:12:08 2022] DELETE /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 04 01:12:08.463019 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-74fa6c82-56cf-4858-99f3-0cbc2906ebd2 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:08.463632 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-74fa6c82-56cf-4858-99f3-0cbc2906ebd2 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: 52505c46-cb03-49ec-a7ba-696d7c6ac219] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:08.472429 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-74fa6c82-56cf-4858-99f3-0cbc2906ebd2 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:08.472915 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-74fa6c82-56cf-4858-99f3-0cbc2906ebd2 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:08.473376 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-74fa6c82-56cf-4858-99f3-0cbc2906ebd2 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:08.474067 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-74fa6c82-56cf-4858-99f3-0cbc2906ebd2 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:08.474517 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-74fa6c82-56cf-4858-99f3-0cbc2906ebd2 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:08.474966 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-74fa6c82-56cf-4858-99f3-0cbc2906ebd2 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:08.488297 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.wsgi [None req-74fa6c82-56cf-4858-99f3-0cbc2906ebd2 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] HTTP exception thrown: Instance 52505c46-cb03-49ec-a7ba-696d7c6ac219 could not be found. Sep 04 01:12:08.489998 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-74fa6c82-56cf-4858-99f3-0cbc2906ebd2 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Request method failure captured: Sep 04 01:12:08.489998 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: request: GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219 HTTP/1.1 Sep 04 01:12:08.489998 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept: application/json Sep 04 01:12:08.489998 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept-Encoding: identity Sep 04 01:12:08.489998 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Connection: close Sep 04 01:12:08.489998 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Length: 0 Sep 04 01:12:08.489998 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Type: application/json Sep 04 01:12:08.489998 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Host: 10.222.0.42 Sep 04 01:12:08.489998 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Openstack-System-Scope: None Sep 04 01:12:08.489998 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: User-Agent: python-urllib3/1.26.12 Sep 04 01:12:08.489998 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Auth-Token: gAAAAABjE_nJhkGHWb7T6kbA22g6MQpQUnzziOpw0WCnFjRF-qzwvonIMqa9XQHefbSvkVL4GGxkzosDVGSxep3ShpT_NSR1Y0CpdR-zYyq1zOcQJ4WQ7o2EXz_hogwf8tYG4DUEXAu4uYN44FQubcx8a7LGqRtbPamZnr7PotOUcFaQv_B5v7E Sep 04 01:12:08.489998 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Id: None Sep 04 01:12:08.489998 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Name: None Sep 04 01:12:08.489998 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Identity-Status: Confirmed Sep 04 01:12:08.489998 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Is-Admin-Project: True Sep 04 01:12:08.489998 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Id: default Sep 04 01:12:08.489998 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Name: Default Sep 04 01:12:08.489998 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Id: e7dca68418a14d7da9aa63ac25f202e8 Sep 04 01:12:08.489998 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Name: tempest-AttachVolumeTestJSON-1676343226 Sep 04 01:12:08.489998 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Role: reader,member Sep 04 01:12:08.489998 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Roles: reader,member Sep 04 01:12:08.491681 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/e7dca68418a14d7da9aa63ac25f202e8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_e7dca68418a14d7da9aa63ac25f202e8", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/e7dca68418a14d7da9aa63ac25f202e8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/e7dca68418a14d7da9aa63ac25f202e8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:12:08.491681 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant: tempest-AttachVolumeTestJSON-1676343226 Sep 04 01:12:08.491681 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Id: e7dca68418a14d7da9aa63ac25f202e8 Sep 04 01:12:08.491681 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1676343226 Sep 04 01:12:08.491681 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User: tempest-AttachVolumeTestJSON-1676343226-project Sep 04 01:12:08.491681 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Id: default Sep 04 01:12:08.491681 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Name: Default Sep 04 01:12:08.491681 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Id: 44993cd93360442bb0afb28778ee83bc Sep 04 01:12:08.491681 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Name: tempest-AttachVolumeTestJSON-1676343226-project Sep 04 01:12:08.491681 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: method: > Sep 04 01:12:08.491681 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: exception: Instance 52505c46-cb03-49ec-a7ba-696d7c6ac219 could not be found. Sep 04 01:12:08.491681 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: action_args: {'id': '52505c46-cb03-49ec-a7ba-696d7c6ac219'} Sep 04 01:12:08.495428 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:12:08.495428 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:08.495428 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:12:08.495428 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:12:08.495428 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:12:08.495428 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:12:08.495428 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:12:08.495428 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:12:08.495428 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:12:08.495428 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:12:08.495428 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 52505c46-cb03-49ec-a7ba-696d7c6ac219 could not be found. Sep 04 01:12:08.495428 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:08.495428 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:08.495428 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:08.495428 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:08.495428 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:08.495428 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:08.495428 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:12:08.495428 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:12:08.495428 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:12:08.496940 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:12:08.496940 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 04 01:12:08.496940 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 04 01:12:08.496940 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:12:08.496940 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:12:08.496940 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:12:08.496940 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:12:08.496940 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 52505c46-cb03-49ec-a7ba-696d7c6ac219 could not be found. Sep 04 01:12:08.496940 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:08.496940 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:08.496940 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:08.496940 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:08.496940 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:08.496940 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:08.496940 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:12:08.496940 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:12:08.496940 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 52505c46-cb03-49ec-a7ba-696d7c6ac219 could not be found. Sep 04 01:12:08.496940 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:08.498293 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-74fa6c82-56cf-4858-99f3-0cbc2906ebd2 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Returning 404 to user: Instance 52505c46-cb03-49ec-a7ba-696d7c6ac219 could not be found. {{(pid=121036) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:12:08.498293 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-74fa6c82-56cf-4858-99f3-0cbc2906ebd2 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219" status: 404 len: 111 microversion: 2.1 time: 0.033682 Sep 04 01:12:08.498293 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 540/1049] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:08 2022] GET /compute/v2.1/servers/52505c46-cb03-49ec-a7ba-696d7c6ac219 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 04 01:12:08.524469 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-ed25a029-3090-49ab-b2db-a67b00a56866 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:08.525209 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-ed25a029-3090-49ab-b2db-a67b00a56866 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:08.531822 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ed25a029-3090-49ab-b2db-a67b00a56866 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:08.532305 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ed25a029-3090-49ab-b2db-a67b00a56866 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:08.543994 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ed25a029-3090-49ab-b2db-a67b00a56866 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:08.543994 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ed25a029-3090-49ab-b2db-a67b00a56866 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:08.543994 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ed25a029-3090-49ab-b2db-a67b00a56866 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:08.543994 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ed25a029-3090-49ab-b2db-a67b00a56866 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:08.662905 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-ed25a029-3090-49ab-b2db-a67b00a56866 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:08.670753 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-ed25a029-3090-49ab-b2db-a67b00a56866 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1568 microversion: 2.1 time: 0.151791 Sep 04 01:12:08.671842 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 510/1050] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:08 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1568 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:12:09.695085 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-be33173b-b968-4704-9f51-da9394b99509 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:09.695694 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-be33173b-b968-4704-9f51-da9394b99509 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:09.702203 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-be33173b-b968-4704-9f51-da9394b99509 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:09.703051 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-be33173b-b968-4704-9f51-da9394b99509 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:09.703512 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-be33173b-b968-4704-9f51-da9394b99509 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:09.704364 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-be33173b-b968-4704-9f51-da9394b99509 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:09.704843 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-be33173b-b968-4704-9f51-da9394b99509 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:09.705293 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-be33173b-b968-4704-9f51-da9394b99509 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:09.818081 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-5bbb7401-fc7f-49bd-bbc5-d2fe2adeb652 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:09.824156 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-5bbb7401-fc7f-49bd-bbc5-d2fe2adeb652 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:09.825014 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5bbb7401-fc7f-49bd-bbc5-d2fe2adeb652 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:09.825567 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5bbb7401-fc7f-49bd-bbc5-d2fe2adeb652 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:09.826008 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5bbb7401-fc7f-49bd-bbc5-d2fe2adeb652 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:09.826719 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5bbb7401-fc7f-49bd-bbc5-d2fe2adeb652 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:09.827174 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5bbb7401-fc7f-49bd-bbc5-d2fe2adeb652 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:09.827627 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5bbb7401-fc7f-49bd-bbc5-d2fe2adeb652 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:09.841614 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.wsgi [None req-5bbb7401-fc7f-49bd-bbc5-d2fe2adeb652 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] HTTP exception thrown: Instance d296b21a-c705-4d0b-8402-19a9c6f2e3d2 could not be found. Sep 04 01:12:09.843755 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-5bbb7401-fc7f-49bd-bbc5-d2fe2adeb652 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Request method failure captured: Sep 04 01:12:09.843755 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: request: DELETE /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 HTTP/1.1 Sep 04 01:12:09.843755 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept: application/json Sep 04 01:12:09.843755 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept-Encoding: identity Sep 04 01:12:09.843755 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Connection: close Sep 04 01:12:09.843755 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Length: 0 Sep 04 01:12:09.843755 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Type: application/json Sep 04 01:12:09.843755 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Host: 10.222.0.42 Sep 04 01:12:09.843755 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Openstack-System-Scope: None Sep 04 01:12:09.843755 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: User-Agent: python-urllib3/1.26.12 Sep 04 01:12:09.843755 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Auth-Token: gAAAAABjE_nJhkGHWb7T6kbA22g6MQpQUnzziOpw0WCnFjRF-qzwvonIMqa9XQHefbSvkVL4GGxkzosDVGSxep3ShpT_NSR1Y0CpdR-zYyq1zOcQJ4WQ7o2EXz_hogwf8tYG4DUEXAu4uYN44FQubcx8a7LGqRtbPamZnr7PotOUcFaQv_B5v7E Sep 04 01:12:09.843755 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Id: None Sep 04 01:12:09.843755 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Name: None Sep 04 01:12:09.843755 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Identity-Status: Confirmed Sep 04 01:12:09.843755 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Is-Admin-Project: True Sep 04 01:12:09.843755 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Id: default Sep 04 01:12:09.843755 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Name: Default Sep 04 01:12:09.843755 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Id: e7dca68418a14d7da9aa63ac25f202e8 Sep 04 01:12:09.843755 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Name: tempest-AttachVolumeTestJSON-1676343226 Sep 04 01:12:09.843755 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Role: reader,member Sep 04 01:12:09.843755 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Roles: reader,member Sep 04 01:12:09.845677 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/e7dca68418a14d7da9aa63ac25f202e8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_e7dca68418a14d7da9aa63ac25f202e8", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/e7dca68418a14d7da9aa63ac25f202e8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/e7dca68418a14d7da9aa63ac25f202e8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:12:09.845677 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant: tempest-AttachVolumeTestJSON-1676343226 Sep 04 01:12:09.845677 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Id: e7dca68418a14d7da9aa63ac25f202e8 Sep 04 01:12:09.845677 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1676343226 Sep 04 01:12:09.845677 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User: tempest-AttachVolumeTestJSON-1676343226-project Sep 04 01:12:09.845677 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Id: default Sep 04 01:12:09.845677 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Name: Default Sep 04 01:12:09.845677 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Id: 44993cd93360442bb0afb28778ee83bc Sep 04 01:12:09.845677 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Name: tempest-AttachVolumeTestJSON-1676343226-project Sep 04 01:12:09.845677 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: method: > Sep 04 01:12:09.845677 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: exception: Instance d296b21a-c705-4d0b-8402-19a9c6f2e3d2 could not be found. Sep 04 01:12:09.845677 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: action_args: {'id': 'd296b21a-c705-4d0b-8402-19a9c6f2e3d2'} Sep 04 01:12:09.846567 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:12:09.846567 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:09.846567 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:12:09.846567 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:12:09.846567 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:12:09.846567 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:12:09.846567 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:12:09.846567 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:12:09.846567 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:12:09.846567 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:12:09.846567 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d296b21a-c705-4d0b-8402-19a9c6f2e3d2 could not be found. Sep 04 01:12:09.846567 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:09.846567 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:09.846567 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:09.846567 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:09.846567 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:09.846567 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:09.846567 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:12:09.846567 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:12:09.846567 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:12:09.847990 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:12:09.847990 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 04 01:12:09.847990 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 04 01:12:09.847990 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 04 01:12:09.847990 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 04 01:12:09.847990 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:12:09.847990 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:12:09.847990 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:12:09.847990 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:12:09.847990 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d296b21a-c705-4d0b-8402-19a9c6f2e3d2 could not be found. Sep 04 01:12:09.847990 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:09.847990 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:09.847990 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:09.847990 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:09.847990 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:09.847990 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:09.847990 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:12:09.847990 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:12:09.847990 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d296b21a-c705-4d0b-8402-19a9c6f2e3d2 could not be found. Sep 04 01:12:09.847990 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:09.850158 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-5bbb7401-fc7f-49bd-bbc5-d2fe2adeb652 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Returning 404 to user: Instance d296b21a-c705-4d0b-8402-19a9c6f2e3d2 could not be found. {{(pid=121037) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:12:09.851192 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-5bbb7401-fc7f-49bd-bbc5-d2fe2adeb652 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "DELETE /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 404 len: 111 microversion: 2.1 time: 0.036966 Sep 04 01:12:09.851757 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 511/1051] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 4 01:12:09 2022] DELETE /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 04 01:12:09.860412 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-74132de5-52a9-4096-a28a-82dba13f1853 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:09.861027 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-74132de5-52a9-4096-a28a-82dba13f1853 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] [instance: d296b21a-c705-4d0b-8402-19a9c6f2e3d2] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:09.865036 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-be33173b-b968-4704-9f51-da9394b99509 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:09.867527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-74132de5-52a9-4096-a28a-82dba13f1853 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:09.868013 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-74132de5-52a9-4096-a28a-82dba13f1853 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:09.868450 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-74132de5-52a9-4096-a28a-82dba13f1853 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:09.869177 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-74132de5-52a9-4096-a28a-82dba13f1853 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:09.869625 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-74132de5-52a9-4096-a28a-82dba13f1853 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:09.870285 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-74132de5-52a9-4096-a28a-82dba13f1853 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:09.877904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-be33173b-b968-4704-9f51-da9394b99509 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1568 microversion: 2.1 time: 0.186664 Sep 04 01:12:09.888700 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 541/1052] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:09 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1568 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:12:09.889054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.wsgi [None req-74132de5-52a9-4096-a28a-82dba13f1853 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] HTTP exception thrown: Instance d296b21a-c705-4d0b-8402-19a9c6f2e3d2 could not be found. Sep 04 01:12:09.889054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-74132de5-52a9-4096-a28a-82dba13f1853 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Request method failure captured: Sep 04 01:12:09.889054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: request: GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 HTTP/1.1 Sep 04 01:12:09.889054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept: application/json Sep 04 01:12:09.889054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept-Encoding: identity Sep 04 01:12:09.889054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Connection: close Sep 04 01:12:09.889054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Length: 0 Sep 04 01:12:09.889054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Type: application/json Sep 04 01:12:09.889054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Host: 10.222.0.42 Sep 04 01:12:09.889054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Openstack-System-Scope: None Sep 04 01:12:09.889054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: User-Agent: python-urllib3/1.26.12 Sep 04 01:12:09.889054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Auth-Token: gAAAAABjE_nJhkGHWb7T6kbA22g6MQpQUnzziOpw0WCnFjRF-qzwvonIMqa9XQHefbSvkVL4GGxkzosDVGSxep3ShpT_NSR1Y0CpdR-zYyq1zOcQJ4WQ7o2EXz_hogwf8tYG4DUEXAu4uYN44FQubcx8a7LGqRtbPamZnr7PotOUcFaQv_B5v7E Sep 04 01:12:09.889054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Id: None Sep 04 01:12:09.889054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Name: None Sep 04 01:12:09.889054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Identity-Status: Confirmed Sep 04 01:12:09.889054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Is-Admin-Project: True Sep 04 01:12:09.889054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Id: default Sep 04 01:12:09.889054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Name: Default Sep 04 01:12:09.889054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Id: e7dca68418a14d7da9aa63ac25f202e8 Sep 04 01:12:09.889054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Name: tempest-AttachVolumeTestJSON-1676343226 Sep 04 01:12:09.889054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Role: reader,member Sep 04 01:12:09.889054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Roles: reader,member Sep 04 01:12:09.890471 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/e7dca68418a14d7da9aa63ac25f202e8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_e7dca68418a14d7da9aa63ac25f202e8", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/e7dca68418a14d7da9aa63ac25f202e8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/e7dca68418a14d7da9aa63ac25f202e8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:12:09.890471 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant: tempest-AttachVolumeTestJSON-1676343226 Sep 04 01:12:09.890471 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Id: e7dca68418a14d7da9aa63ac25f202e8 Sep 04 01:12:09.890471 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1676343226 Sep 04 01:12:09.890471 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User: tempest-AttachVolumeTestJSON-1676343226-project Sep 04 01:12:09.890471 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Id: default Sep 04 01:12:09.890471 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Name: Default Sep 04 01:12:09.890471 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Id: 44993cd93360442bb0afb28778ee83bc Sep 04 01:12:09.890471 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Name: tempest-AttachVolumeTestJSON-1676343226-project Sep 04 01:12:09.890471 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: method: > Sep 04 01:12:09.890471 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: exception: Instance d296b21a-c705-4d0b-8402-19a9c6f2e3d2 could not be found. Sep 04 01:12:09.890471 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: action_args: {'id': 'd296b21a-c705-4d0b-8402-19a9c6f2e3d2'} Sep 04 01:12:09.897645 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:12:09.897645 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:09.897645 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:12:09.897645 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:12:09.897645 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:12:09.897645 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:12:09.897645 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:12:09.897645 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:12:09.897645 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:12:09.897645 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:12:09.897645 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d296b21a-c705-4d0b-8402-19a9c6f2e3d2 could not be found. Sep 04 01:12:09.897645 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:09.897645 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:09.897645 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:09.897645 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:09.897645 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:09.897645 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:09.897645 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:12:09.897645 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:12:09.897645 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:12:09.899513 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:12:09.899513 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 04 01:12:09.899513 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 04 01:12:09.899513 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:12:09.899513 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:12:09.899513 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:12:09.899513 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:12:09.899513 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d296b21a-c705-4d0b-8402-19a9c6f2e3d2 could not be found. Sep 04 01:12:09.899513 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:09.899513 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:09.899513 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:09.899513 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:09.899513 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:09.899513 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:09.899513 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:12:09.899513 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:12:09.899513 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d296b21a-c705-4d0b-8402-19a9c6f2e3d2 could not be found. Sep 04 01:12:09.899513 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:09.900834 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-74132de5-52a9-4096-a28a-82dba13f1853 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] Returning 404 to user: Instance d296b21a-c705-4d0b-8402-19a9c6f2e3d2 could not be found. {{(pid=121037) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:12:09.900834 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-74132de5-52a9-4096-a28a-82dba13f1853 tempest-AttachVolumeTestJSON-1676343226 tempest-AttachVolumeTestJSON-1676343226-project] 10.222.0.42 "GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2" status: 404 len: 111 microversion: 2.1 time: 0.031573 Sep 04 01:12:09.900834 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 512/1053] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:09 2022] GET /compute/v2.1/servers/d296b21a-c705-4d0b-8402-19a9c6f2e3d2 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 04 01:12:10.909532 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-e27e7496-febd-4b5e-a4de-39d108cf89f2 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:10.910216 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-e27e7496-febd-4b5e-a4de-39d108cf89f2 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:10.916657 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e27e7496-febd-4b5e-a4de-39d108cf89f2 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:10.917161 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e27e7496-febd-4b5e-a4de-39d108cf89f2 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:10.917606 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e27e7496-febd-4b5e-a4de-39d108cf89f2 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:10.918307 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e27e7496-febd-4b5e-a4de-39d108cf89f2 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:10.918768 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e27e7496-febd-4b5e-a4de-39d108cf89f2 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:10.919196 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e27e7496-febd-4b5e-a4de-39d108cf89f2 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:11.152907 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e27e7496-febd-4b5e-a4de-39d108cf89f2 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:11.163509 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-e27e7496-febd-4b5e-a4de-39d108cf89f2 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1569 microversion: 2.1 time: 0.255924 Sep 04 01:12:11.163509 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 542/1054] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:10 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1569 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:12:11.184822 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-c65a9e37-cc1d-449d-8e60-4a0eae93ee37 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:12:11.186115 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-c65a9e37-cc1d-449d-8e60-4a0eae93ee37 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:11.192915 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c65a9e37-cc1d-449d-8e60-4a0eae93ee37 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:11.193858 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c65a9e37-cc1d-449d-8e60-4a0eae93ee37 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:11.193858 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c65a9e37-cc1d-449d-8e60-4a0eae93ee37 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:11.194537 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c65a9e37-cc1d-449d-8e60-4a0eae93ee37 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:11.194998 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c65a9e37-cc1d-449d-8e60-4a0eae93ee37 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:11.195431 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c65a9e37-cc1d-449d-8e60-4a0eae93ee37 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:11.225727 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-c65a9e37-cc1d-449d-8e60-4a0eae93ee37 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=121037) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Sep 04 01:12:11.543776 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-c65a9e37-cc1d-449d-8e60-4a0eae93ee37 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "POST /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff/action" status: 202 len: 0 microversion: 2.1 time: 0.362808 Sep 04 01:12:11.544702 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 513/1055] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Sep 4 01:12:11 2022] POST /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff/action => generated 0 bytes in 364 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 04 01:12:11.553282 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-8627c4bf-7fcd-4a57-ba61-b1c519503d87 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:11.554081 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-8627c4bf-7fcd-4a57-ba61-b1c519503d87 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:11.561450 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8627c4bf-7fcd-4a57-ba61-b1c519503d87 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:11.561901 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8627c4bf-7fcd-4a57-ba61-b1c519503d87 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:11.562433 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8627c4bf-7fcd-4a57-ba61-b1c519503d87 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:11.563352 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8627c4bf-7fcd-4a57-ba61-b1c519503d87 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:11.564034 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8627c4bf-7fcd-4a57-ba61-b1c519503d87 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:11.565276 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8627c4bf-7fcd-4a57-ba61-b1c519503d87 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:11.859577 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-8627c4bf-7fcd-4a57-ba61-b1c519503d87 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:11.870230 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-8627c4bf-7fcd-4a57-ba61-b1c519503d87 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1517 microversion: 2.1 time: 0.321064 Sep 04 01:12:11.873752 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 543/1056] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:11 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1517 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:12:12.708769 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-ea2c95fb-4142-4914-a56a-f293ff0c9189 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:12.709344 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-ea2c95fb-4142-4914-a56a-f293ff0c9189 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 2718d3b6-c71c-4ae1-9544-0f8e4a43d989] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:12.716912 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea2c95fb-4142-4914-a56a-f293ff0c9189 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:12.717442 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea2c95fb-4142-4914-a56a-f293ff0c9189 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:12.717880 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea2c95fb-4142-4914-a56a-f293ff0c9189 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:12.718899 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea2c95fb-4142-4914-a56a-f293ff0c9189 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:12.719359 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea2c95fb-4142-4914-a56a-f293ff0c9189 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:12.719787 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea2c95fb-4142-4914-a56a-f293ff0c9189 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:12.764478 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-ea2c95fb-4142-4914-a56a-f293ff0c9189 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989/ips" status: 200 len: 151 microversion: 2.1 time: 0.059669 Sep 04 01:12:12.767348 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 514/1057] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Sep 4 01:12:12 2022] GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989/ips => generated 151 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:12:12.896863 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-9b97cbbe-e9a7-4d6a-997b-422fcdc51828 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:12.898193 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-9b97cbbe-e9a7-4d6a-997b-422fcdc51828 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:12.906220 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9b97cbbe-e9a7-4d6a-997b-422fcdc51828 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:12.906707 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9b97cbbe-e9a7-4d6a-997b-422fcdc51828 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:12.907460 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9b97cbbe-e9a7-4d6a-997b-422fcdc51828 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:12.908172 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9b97cbbe-e9a7-4d6a-997b-422fcdc51828 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:12.908629 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9b97cbbe-e9a7-4d6a-997b-422fcdc51828 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:12.909056 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9b97cbbe-e9a7-4d6a-997b-422fcdc51828 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:13.196346 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-9b97cbbe-e9a7-4d6a-997b-422fcdc51828 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:13.203296 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=100 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:12:13.206626 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-9b97cbbe-e9a7-4d6a-997b-422fcdc51828 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1519 microversion: 2.1 time: 0.314903 Sep 04 01:12:13.207723 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 544/1058] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:12 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1519 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:12:13.926742 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-de0b89d5-f2db-4441-b7bf-66c22a4af795 req-54b9a114-5514-43ee-9f21-b74c6006cb30 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "74df11dc-31bd-4745-83bc-4b3c430d02ff", "name": "network-vif-unplugged", "status": "completed", "tag": "d072bfcf-41bd-430e-800c-1d1a3b17d915"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:12:13.948139 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-de0b89d5-f2db-4441-b7bf-66c22a4af795 req-54b9a114-5514-43ee-9f21-b74c6006cb30 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:13.948672 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-de0b89d5-f2db-4441-b7bf-66c22a4af795 req-54b9a114-5514-43ee-9f21-b74c6006cb30 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:13.949210 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-de0b89d5-f2db-4441-b7bf-66c22a4af795 req-54b9a114-5514-43ee-9f21-b74c6006cb30 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:13.981583 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.compute.server_external_events [req-de0b89d5-f2db-4441-b7bf-66c22a4af795 req-54b9a114-5514-43ee-9f21-b74c6006cb30 service nova] Unable to find a host for instance 74df11dc-31bd-4745-83bc-4b3c430d02ff. Dropping event network-vif-unplugged {{(pid=121037) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:144}} Sep 04 01:12:13.982579 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-de0b89d5-f2db-4441-b7bf-66c22a4af795 req-54b9a114-5514-43ee-9f21-b74c6006cb30 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.060136 Sep 04 01:12:13.983458 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 515/1059] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:12:13 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 61 msecs (HTTP/1.1 207) 9 headers in 367 bytes (1 switches on core 0) Sep 04 01:12:14.232893 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-83ae140e-e8d7-4903-8cff-80346ff1e30f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:14.233483 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-83ae140e-e8d7-4903-8cff-80346ff1e30f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:14.242474 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-83ae140e-e8d7-4903-8cff-80346ff1e30f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:14.242978 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-83ae140e-e8d7-4903-8cff-80346ff1e30f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:14.243419 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-83ae140e-e8d7-4903-8cff-80346ff1e30f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:14.244346 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-83ae140e-e8d7-4903-8cff-80346ff1e30f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:14.244967 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-83ae140e-e8d7-4903-8cff-80346ff1e30f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:14.245597 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-83ae140e-e8d7-4903-8cff-80346ff1e30f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:14.501125 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-83ae140e-e8d7-4903-8cff-80346ff1e30f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:14.512743 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-83ae140e-e8d7-4903-8cff-80346ff1e30f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1519 microversion: 2.1 time: 0.284039 Sep 04 01:12:14.513951 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 545/1060] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:14 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1519 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:12:14.544168 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-293aba57-9e03-49d2-b666-7c0b1e5743e4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "fc89575e-da83-4189-a5ac-08e825d886e7"}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:12:14.547724 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-293aba57-9e03-49d2-b666-7c0b1e5743e4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 2718d3b6-c71c-4ae1-9544-0f8e4a43d989] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:14.554010 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-293aba57-9e03-49d2-b666-7c0b1e5743e4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:14.554746 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-293aba57-9e03-49d2-b666-7c0b1e5743e4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:14.555333 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-293aba57-9e03-49d2-b666-7c0b1e5743e4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:14.556374 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-293aba57-9e03-49d2-b666-7c0b1e5743e4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:14.556973 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-293aba57-9e03-49d2-b666-7c0b1e5743e4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:14.557567 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-293aba57-9e03-49d2-b666-7c0b1e5743e4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:15.005831 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:12:15.534651 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-293aba57-9e03-49d2-b666-7c0b1e5743e4 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "POST /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.996827 Sep 04 01:12:15.535374 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 516/1061] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Sep 4 01:12:14 2022] POST /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989/os-volume_attachments => generated 194 bytes in 998 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:12:15.536738 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-c56b4d32-9eec-443d-b616-7f36a3cf3b4b tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:15.537892 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-c56b4d32-9eec-443d-b616-7f36a3cf3b4b tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:15.540431 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_api:SELECT=61,nova_api:DELETE=1 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:12:15.552680 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c56b4d32-9eec-443d-b616-7f36a3cf3b4b tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:15.553255 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c56b4d32-9eec-443d-b616-7f36a3cf3b4b tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:15.553712 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c56b4d32-9eec-443d-b616-7f36a3cf3b4b tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:15.554443 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c56b4d32-9eec-443d-b616-7f36a3cf3b4b tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:15.554905 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c56b4d32-9eec-443d-b616-7f36a3cf3b4b tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:15.555337 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-c56b4d32-9eec-443d-b616-7f36a3cf3b4b tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:15.721029 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-c56b4d32-9eec-443d-b616-7f36a3cf3b4b tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:15.728826 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-c56b4d32-9eec-443d-b616-7f36a3cf3b4b tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1575 microversion: 2.1 time: 0.196043 Sep 04 01:12:15.729426 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 546/1062] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:15 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1575 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:12:16.755640 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-9d8a21de-0531-48b1-ab00-8c93b3d3f67e tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:16.756337 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-9d8a21de-0531-48b1-ab00-8c93b3d3f67e tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:16.763389 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9d8a21de-0531-48b1-ab00-8c93b3d3f67e tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:16.763896 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9d8a21de-0531-48b1-ab00-8c93b3d3f67e tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:16.764327 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9d8a21de-0531-48b1-ab00-8c93b3d3f67e tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:16.765018 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9d8a21de-0531-48b1-ab00-8c93b3d3f67e tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:16.765511 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9d8a21de-0531-48b1-ab00-8c93b3d3f67e tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:16.765928 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9d8a21de-0531-48b1-ab00-8c93b3d3f67e tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:16.915719 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-9d8a21de-0531-48b1-ab00-8c93b3d3f67e tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:16.925283 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-9d8a21de-0531-48b1-ab00-8c93b3d3f67e tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1575 microversion: 2.1 time: 0.173915 Sep 04 01:12:16.926434 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 517/1063] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:16 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1575 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:12:17.949194 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-f0f5ead2-58c6-4533-8340-b5ec142ff4f7 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:17.949825 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-f0f5ead2-58c6-4533-8340-b5ec142ff4f7 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:17.957384 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f0f5ead2-58c6-4533-8340-b5ec142ff4f7 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:17.957863 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f0f5ead2-58c6-4533-8340-b5ec142ff4f7 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:17.958293 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f0f5ead2-58c6-4533-8340-b5ec142ff4f7 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:17.958977 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f0f5ead2-58c6-4533-8340-b5ec142ff4f7 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:17.959421 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f0f5ead2-58c6-4533-8340-b5ec142ff4f7 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:17.959843 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f0f5ead2-58c6-4533-8340-b5ec142ff4f7 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:18.102942 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-f0f5ead2-58c6-4533-8340-b5ec142ff4f7 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:18.111691 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-f0f5ead2-58c6-4533-8340-b5ec142ff4f7 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1575 microversion: 2.1 time: 0.166361 Sep 04 01:12:18.112262 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 547/1064] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:17 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1575 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:12:19.135526 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-98777757-a49a-4615-8574-f49dbe981654 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:19.136062 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-98777757-a49a-4615-8574-f49dbe981654 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:19.143252 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-98777757-a49a-4615-8574-f49dbe981654 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:19.143750 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-98777757-a49a-4615-8574-f49dbe981654 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:19.144175 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-98777757-a49a-4615-8574-f49dbe981654 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:19.144890 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-98777757-a49a-4615-8574-f49dbe981654 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:19.145357 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-98777757-a49a-4615-8574-f49dbe981654 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:19.145776 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-98777757-a49a-4615-8574-f49dbe981654 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:19.279888 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-98777757-a49a-4615-8574-f49dbe981654 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:19.287876 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-98777757-a49a-4615-8574-f49dbe981654 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1575 microversion: 2.1 time: 0.156701 Sep 04 01:12:19.288430 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 518/1065] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:19 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1575 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:12:20.311533 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-5678beb8-e7af-45fc-9052-f2b044b5feed tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:20.312211 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-5678beb8-e7af-45fc-9052-f2b044b5feed tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:20.318836 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5678beb8-e7af-45fc-9052-f2b044b5feed tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:20.319314 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5678beb8-e7af-45fc-9052-f2b044b5feed tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:20.319747 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5678beb8-e7af-45fc-9052-f2b044b5feed tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:20.320435 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5678beb8-e7af-45fc-9052-f2b044b5feed tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:20.320887 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5678beb8-e7af-45fc-9052-f2b044b5feed tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:20.321346 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5678beb8-e7af-45fc-9052-f2b044b5feed tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:20.463829 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-5678beb8-e7af-45fc-9052-f2b044b5feed tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:20.472344 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-5678beb8-e7af-45fc-9052-f2b044b5feed tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1575 microversion: 2.1 time: 0.164748 Sep 04 01:12:20.472908 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 548/1066] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:20 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1575 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:12:21.496128 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-874c4cd8-8ed0-40e3-8620-9fbc22827f22 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:21.497214 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-874c4cd8-8ed0-40e3-8620-9fbc22827f22 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:21.503929 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-874c4cd8-8ed0-40e3-8620-9fbc22827f22 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:21.504418 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-874c4cd8-8ed0-40e3-8620-9fbc22827f22 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:21.504855 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-874c4cd8-8ed0-40e3-8620-9fbc22827f22 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:21.505580 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-874c4cd8-8ed0-40e3-8620-9fbc22827f22 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:21.506033 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-874c4cd8-8ed0-40e3-8620-9fbc22827f22 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:21.506461 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-874c4cd8-8ed0-40e3-8620-9fbc22827f22 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:21.638803 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-874c4cd8-8ed0-40e3-8620-9fbc22827f22 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:21.647091 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-874c4cd8-8ed0-40e3-8620-9fbc22827f22 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1575 microversion: 2.1 time: 0.154776 Sep 04 01:12:21.647739 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 519/1067] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:21 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1575 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:12:22.670003 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-efb917a3-aa26-4693-a1b8-3231a6fee2a1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:22.670660 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-efb917a3-aa26-4693-a1b8-3231a6fee2a1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:22.677377 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-efb917a3-aa26-4693-a1b8-3231a6fee2a1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:22.677853 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-efb917a3-aa26-4693-a1b8-3231a6fee2a1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:22.678282 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-efb917a3-aa26-4693-a1b8-3231a6fee2a1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:22.678962 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-efb917a3-aa26-4693-a1b8-3231a6fee2a1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:22.679409 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-efb917a3-aa26-4693-a1b8-3231a6fee2a1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:22.679828 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-efb917a3-aa26-4693-a1b8-3231a6fee2a1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:22.814936 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-efb917a3-aa26-4693-a1b8-3231a6fee2a1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:22.822760 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-efb917a3-aa26-4693-a1b8-3231a6fee2a1 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1575 microversion: 2.1 time: 0.156756 Sep 04 01:12:22.823329 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 549/1068] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:22 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1575 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:12:23.191477 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-8dac256a-b5a6-4ea3-8a56-0e5cee5b31b2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:23.192029 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-8dac256a-b5a6-4ea3-8a56-0e5cee5b31b2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 2718d3b6-c71c-4ae1-9544-0f8e4a43d989] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:23.194601 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_api:SELECT=69,nova_api:UPDATE=2 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:12:23.199280 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8dac256a-b5a6-4ea3-8a56-0e5cee5b31b2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:23.199767 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8dac256a-b5a6-4ea3-8a56-0e5cee5b31b2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:23.200204 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8dac256a-b5a6-4ea3-8a56-0e5cee5b31b2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:23.200890 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8dac256a-b5a6-4ea3-8a56-0e5cee5b31b2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:23.201363 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8dac256a-b5a6-4ea3-8a56-0e5cee5b31b2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:23.201820 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8dac256a-b5a6-4ea3-8a56-0e5cee5b31b2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:23.285910 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.objects.instance [None req-8dac256a-b5a6-4ea3-8a56-0e5cee5b31b2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lazy-loading 'services' on Instance uuid 2718d3b6-c71c-4ae1-9544-0f8e4a43d989 {{(pid=121037) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:12:23.454889 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-8dac256a-b5a6-4ea3-8a56-0e5cee5b31b2 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "DELETE /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989/os-volume_attachments/fc89575e-da83-4189-a5ac-08e825d886e7" status: 202 len: 0 microversion: 2.1 time: 0.267974 Sep 04 01:12:23.455479 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 520/1069] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Sep 4 01:12:23 2022] DELETE /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989/os-volume_attachments/fc89575e-da83-4189-a5ac-08e825d886e7 => generated 0 bytes in 269 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 04 01:12:23.464371 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-2e5ae040-bef2-457d-befd-c90c5c0b3659 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:23.466483 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-2e5ae040-bef2-457d-befd-c90c5c0b3659 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 2718d3b6-c71c-4ae1-9544-0f8e4a43d989] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:23.472681 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2e5ae040-bef2-457d-befd-c90c5c0b3659 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:23.473193 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2e5ae040-bef2-457d-befd-c90c5c0b3659 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:23.473627 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2e5ae040-bef2-457d-befd-c90c5c0b3659 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:23.474320 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2e5ae040-bef2-457d-befd-c90c5c0b3659 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:23.474764 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2e5ae040-bef2-457d-befd-c90c5c0b3659 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:23.475186 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2e5ae040-bef2-457d-befd-c90c5c0b3659 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:23.506069 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-2e5ae040-bef2-457d-befd-c90c5c0b3659 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045847 Sep 04 01:12:23.506644 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 550/1070] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 4 01:12:23 2022] GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:12:23.850970 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-354dd4c8-e3f0-4dda-abad-d6a137427fdb tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:23.852112 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-354dd4c8-e3f0-4dda-abad-d6a137427fdb tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:23.859318 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-354dd4c8-e3f0-4dda-abad-d6a137427fdb tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:23.860205 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-354dd4c8-e3f0-4dda-abad-d6a137427fdb tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:23.860772 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-354dd4c8-e3f0-4dda-abad-d6a137427fdb tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:23.861576 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-354dd4c8-e3f0-4dda-abad-d6a137427fdb tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:23.862107 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-354dd4c8-e3f0-4dda-abad-d6a137427fdb tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:23.862613 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-354dd4c8-e3f0-4dda-abad-d6a137427fdb tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:24.008872 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-354dd4c8-e3f0-4dda-abad-d6a137427fdb tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:24.017867 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-354dd4c8-e3f0-4dda-abad-d6a137427fdb tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1575 microversion: 2.1 time: 0.173072 Sep 04 01:12:24.018444 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 521/1071] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:23 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1575 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:12:24.519250 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-f4d1c769-dea1-41d7-895c-d77692ea1cf1 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:24.521516 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-f4d1c769-dea1-41d7-895c-d77692ea1cf1 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 2718d3b6-c71c-4ae1-9544-0f8e4a43d989] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:24.528171 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f4d1c769-dea1-41d7-895c-d77692ea1cf1 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:24.528647 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f4d1c769-dea1-41d7-895c-d77692ea1cf1 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:24.529097 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f4d1c769-dea1-41d7-895c-d77692ea1cf1 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:24.529789 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f4d1c769-dea1-41d7-895c-d77692ea1cf1 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:24.530263 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f4d1c769-dea1-41d7-895c-d77692ea1cf1 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:24.530761 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f4d1c769-dea1-41d7-895c-d77692ea1cf1 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:24.561118 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-f4d1c769-dea1-41d7-895c-d77692ea1cf1 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046612 Sep 04 01:12:24.562164 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 551/1072] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 4 01:12:24 2022] GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:12:25.041183 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-72d2191a-f131-4537-a6c0-3be8ae22bb85 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:25.041710 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-72d2191a-f131-4537-a6c0-3be8ae22bb85 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:25.048279 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-72d2191a-f131-4537-a6c0-3be8ae22bb85 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:25.048748 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-72d2191a-f131-4537-a6c0-3be8ae22bb85 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:25.049207 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-72d2191a-f131-4537-a6c0-3be8ae22bb85 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:25.049885 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-72d2191a-f131-4537-a6c0-3be8ae22bb85 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:25.050349 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-72d2191a-f131-4537-a6c0-3be8ae22bb85 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:25.050748 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-72d2191a-f131-4537-a6c0-3be8ae22bb85 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:25.182434 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-72d2191a-f131-4537-a6c0-3be8ae22bb85 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:25.190432 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-72d2191a-f131-4537-a6c0-3be8ae22bb85 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1575 microversion: 2.1 time: 0.153183 Sep 04 01:12:25.190996 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 522/1073] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:25 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1575 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:12:25.574261 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-723eb4f6-c7e2-41e9-bd2e-75c8ae0e3801 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:25.576419 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-723eb4f6-c7e2-41e9-bd2e-75c8ae0e3801 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 2718d3b6-c71c-4ae1-9544-0f8e4a43d989] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:25.583158 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-723eb4f6-c7e2-41e9-bd2e-75c8ae0e3801 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:25.583642 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-723eb4f6-c7e2-41e9-bd2e-75c8ae0e3801 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:25.584188 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-723eb4f6-c7e2-41e9-bd2e-75c8ae0e3801 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:25.584891 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-723eb4f6-c7e2-41e9-bd2e-75c8ae0e3801 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:25.585370 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-723eb4f6-c7e2-41e9-bd2e-75c8ae0e3801 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:25.585796 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-723eb4f6-c7e2-41e9-bd2e-75c8ae0e3801 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:25.616952 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-723eb4f6-c7e2-41e9-bd2e-75c8ae0e3801 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.046831 Sep 04 01:12:25.622603 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 552/1074] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 4 01:12:25 2022] GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989/os-volume_attachments => generated 25 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 04 01:12:25.686818 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-fd4b9d32-715f-4f3b-9775-685b8c1c8f8e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:25.687304 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-fd4b9d32-715f-4f3b-9775-685b8c1c8f8e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 2718d3b6-c71c-4ae1-9544-0f8e4a43d989] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:25.693364 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fd4b9d32-715f-4f3b-9775-685b8c1c8f8e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:25.693841 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fd4b9d32-715f-4f3b-9775-685b8c1c8f8e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:25.694274 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fd4b9d32-715f-4f3b-9775-685b8c1c8f8e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:25.694951 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fd4b9d32-715f-4f3b-9775-685b8c1c8f8e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:25.695398 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fd4b9d32-715f-4f3b-9775-685b8c1c8f8e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:25.695816 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fd4b9d32-715f-4f3b-9775-685b8c1c8f8e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:25.725989 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-fd4b9d32-715f-4f3b-9775-685b8c1c8f8e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 2718d3b6-c71c-4ae1-9544-0f8e4a43d989] Going to try to terminate instance {{(pid=121037) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 04 01:12:26.012839 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-3fd0981e-7f61-4495-84d1-108ed19f74cb req-b30b23c2-0025-4621-9951-6fa1bbaa9260 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "74df11dc-31bd-4745-83bc-4b3c430d02ff", "name": "network-vif-plugged", "status": "completed", "tag": "d072bfcf-41bd-430e-800c-1d1a3b17d915"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:12:26.015468 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-fd4b9d32-715f-4f3b-9775-685b8c1c8f8e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "DELETE /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989" status: 204 len: 0 microversion: 2.1 time: 0.332113 Sep 04 01:12:26.016898 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 523/1075] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 4 01:12:25 2022] DELETE /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 04 01:12:26.024834 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-3fd0981e-7f61-4495-84d1-108ed19f74cb req-b30b23c2-0025-4621-9951-6fa1bbaa9260 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:26.025339 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-3fd0981e-7f61-4495-84d1-108ed19f74cb req-b30b23c2-0025-4621-9951-6fa1bbaa9260 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:26.025769 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-3fd0981e-7f61-4495-84d1-108ed19f74cb req-b30b23c2-0025-4621-9951-6fa1bbaa9260 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:26.026999 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-6600047a-90dc-48a2-8529-2d802f3912bb tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:26.027743 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-6600047a-90dc-48a2-8529-2d802f3912bb tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 2718d3b6-c71c-4ae1-9544-0f8e4a43d989] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:26.037382 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6600047a-90dc-48a2-8529-2d802f3912bb tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:26.039152 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6600047a-90dc-48a2-8529-2d802f3912bb tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:26.039152 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6600047a-90dc-48a2-8529-2d802f3912bb tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:26.039152 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6600047a-90dc-48a2-8529-2d802f3912bb tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:26.039592 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6600047a-90dc-48a2-8529-2d802f3912bb tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:26.039798 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6600047a-90dc-48a2-8529-2d802f3912bb tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:26.040905 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-3fd0981e-7f61-4495-84d1-108ed19f74cb req-b30b23c2-0025-4621-9951-6fa1bbaa9260 service nova] Creating event network-vif-plugged:d072bfcf-41bd-430e-800c-1d1a3b17d915 for instance 74df11dc-31bd-4745-83bc-4b3c430d02ff on hv-cinder-maste Sep 04 01:12:26.047361 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-3fd0981e-7f61-4495-84d1-108ed19f74cb req-b30b23c2-0025-4621-9951-6fa1bbaa9260 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038492 Sep 04 01:12:26.048305 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 553/1076] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:12:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:12:26.179722 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-6600047a-90dc-48a2-8529-2d802f3912bb tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:26.188150 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-6600047a-90dc-48a2-8529-2d802f3912bb tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989" status: 200 len: 1682 microversion: 2.1 time: 0.165026 Sep 04 01:12:26.188710 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 524/1077] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:26 2022] GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989 => generated 1682 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:12:26.214387 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-ceef7f89-2b2c-4ba0-b8e3-4a3d34c6f80d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:26.215145 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-ceef7f89-2b2c-4ba0-b8e3-4a3d34c6f80d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:26.221456 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ceef7f89-2b2c-4ba0-b8e3-4a3d34c6f80d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:26.221933 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ceef7f89-2b2c-4ba0-b8e3-4a3d34c6f80d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:26.222369 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ceef7f89-2b2c-4ba0-b8e3-4a3d34c6f80d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:26.223057 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ceef7f89-2b2c-4ba0-b8e3-4a3d34c6f80d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:26.223510 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ceef7f89-2b2c-4ba0-b8e3-4a3d34c6f80d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:26.224056 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ceef7f89-2b2c-4ba0-b8e3-4a3d34c6f80d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:26.363004 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ceef7f89-2b2c-4ba0-b8e3-4a3d34c6f80d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:26.371251 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-ceef7f89-2b2c-4ba0-b8e3-4a3d34c6f80d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1575 microversion: 2.1 time: 0.161114 Sep 04 01:12:26.371869 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 554/1078] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:26 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1575 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:12:27.212513 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-79b0fa4b-6d7d-4d4c-8957-df68de08aa83 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:27.213147 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-79b0fa4b-6d7d-4d4c-8957-df68de08aa83 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 2718d3b6-c71c-4ae1-9544-0f8e4a43d989] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:27.219754 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-79b0fa4b-6d7d-4d4c-8957-df68de08aa83 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:27.220227 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-79b0fa4b-6d7d-4d4c-8957-df68de08aa83 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:27.220658 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-79b0fa4b-6d7d-4d4c-8957-df68de08aa83 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:27.221361 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-79b0fa4b-6d7d-4d4c-8957-df68de08aa83 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:27.221814 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-79b0fa4b-6d7d-4d4c-8957-df68de08aa83 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:27.222230 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-79b0fa4b-6d7d-4d4c-8957-df68de08aa83 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:27.394649 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-79b0fa4b-6d7d-4d4c-8957-df68de08aa83 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:27.395222 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-1d3d0b58-449b-40bb-9d16-694d0674a79d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:27.395944 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-1d3d0b58-449b-40bb-9d16-694d0674a79d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:27.402575 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1d3d0b58-449b-40bb-9d16-694d0674a79d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:27.403062 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1d3d0b58-449b-40bb-9d16-694d0674a79d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:27.403190 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-79b0fa4b-6d7d-4d4c-8957-df68de08aa83 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989" status: 200 len: 1682 microversion: 2.1 time: 0.194587 Sep 04 01:12:27.403499 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1d3d0b58-449b-40bb-9d16-694d0674a79d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:27.417681 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 525/1079] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:27 2022] GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989 => generated 1682 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:12:27.417907 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1d3d0b58-449b-40bb-9d16-694d0674a79d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:27.417907 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1d3d0b58-449b-40bb-9d16-694d0674a79d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:27.417907 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1d3d0b58-449b-40bb-9d16-694d0674a79d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:27.823588 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-1d3d0b58-449b-40bb-9d16-694d0674a79d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:27.841377 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-1d3d0b58-449b-40bb-9d16-694d0674a79d tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1575 microversion: 2.1 time: 0.444438 Sep 04 01:12:27.841377 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 555/1080] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:27 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1575 bytes in 446 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:12:28.429892 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-a10646b6-9452-4b7f-93ff-e0ff9fe72a94 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:28.430809 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-a10646b6-9452-4b7f-93ff-e0ff9fe72a94 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 2718d3b6-c71c-4ae1-9544-0f8e4a43d989] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:28.438768 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a10646b6-9452-4b7f-93ff-e0ff9fe72a94 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:28.439269 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a10646b6-9452-4b7f-93ff-e0ff9fe72a94 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:28.442459 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a10646b6-9452-4b7f-93ff-e0ff9fe72a94 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:28.442459 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a10646b6-9452-4b7f-93ff-e0ff9fe72a94 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:28.442459 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a10646b6-9452-4b7f-93ff-e0ff9fe72a94 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:28.442459 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a10646b6-9452-4b7f-93ff-e0ff9fe72a94 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:28.707376 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-a10646b6-9452-4b7f-93ff-e0ff9fe72a94 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:28.714356 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-a10646b6-9452-4b7f-93ff-e0ff9fe72a94 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989" status: 200 len: 1615 microversion: 2.1 time: 0.288535 Sep 04 01:12:28.714927 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 526/1081] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:28 2022] GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989 => generated 1615 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:12:28.863060 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-09d92773-88e0-490a-825a-4ae0369892be tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:28.863972 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-09d92773-88e0-490a-825a-4ae0369892be tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:28.870633 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-09d92773-88e0-490a-825a-4ae0369892be tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:28.871117 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-09d92773-88e0-490a-825a-4ae0369892be tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:28.871547 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-09d92773-88e0-490a-825a-4ae0369892be tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:28.872238 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-09d92773-88e0-490a-825a-4ae0369892be tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:28.872701 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-09d92773-88e0-490a-825a-4ae0369892be tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:28.873127 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-09d92773-88e0-490a-825a-4ae0369892be tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:29.018341 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-09d92773-88e0-490a-825a-4ae0369892be tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:29.034571 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-09d92773-88e0-490a-825a-4ae0369892be tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1575 microversion: 2.1 time: 0.178540 Sep 04 01:12:29.035856 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 556/1082] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:28 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1575 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:12:29.095043 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-24271ad5-6917-422b-b1e0-71eaf12edb8b req-8387fc45-45a6-4be6-bd67-26c7ad4ccec3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2718d3b6-c71c-4ae1-9544-0f8e4a43d989", "name": "network-vif-deleted", "tag": "a684da0f-7a2f-4974-8aa3-30875caa5a0e"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:12:29.111524 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-24271ad5-6917-422b-b1e0-71eaf12edb8b req-8387fc45-45a6-4be6-bd67-26c7ad4ccec3 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:29.112152 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-24271ad5-6917-422b-b1e0-71eaf12edb8b req-8387fc45-45a6-4be6-bd67-26c7ad4ccec3 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:29.112841 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-24271ad5-6917-422b-b1e0-71eaf12edb8b req-8387fc45-45a6-4be6-bd67-26c7ad4ccec3 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:29.130068 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-24271ad5-6917-422b-b1e0-71eaf12edb8b req-8387fc45-45a6-4be6-bd67-26c7ad4ccec3 service nova] Creating event network-vif-deleted:a684da0f-7a2f-4974-8aa3-30875caa5a0e for instance 2718d3b6-c71c-4ae1-9544-0f8e4a43d989 on hv-cinder-maste Sep 04 01:12:29.135837 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-24271ad5-6917-422b-b1e0-71eaf12edb8b req-8387fc45-45a6-4be6-bd67-26c7ad4ccec3 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046256 Sep 04 01:12:29.136853 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 527/1083] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:12:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:12:29.738924 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-518e912f-17aa-4808-a3a1-ce1c561b9dfc tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:29.739532 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-518e912f-17aa-4808-a3a1-ce1c561b9dfc tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 2718d3b6-c71c-4ae1-9544-0f8e4a43d989] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:29.746277 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-518e912f-17aa-4808-a3a1-ce1c561b9dfc tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:29.747642 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-518e912f-17aa-4808-a3a1-ce1c561b9dfc tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:29.747642 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-518e912f-17aa-4808-a3a1-ce1c561b9dfc tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:29.747975 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-518e912f-17aa-4808-a3a1-ce1c561b9dfc tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:29.748499 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-518e912f-17aa-4808-a3a1-ce1c561b9dfc tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:29.757379 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-518e912f-17aa-4808-a3a1-ce1c561b9dfc tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:29.796546 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-518e912f-17aa-4808-a3a1-ce1c561b9dfc tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:29.797106 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-518e912f-17aa-4808-a3a1-ce1c561b9dfc tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:29.797597 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-518e912f-17aa-4808-a3a1-ce1c561b9dfc tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:29.798053 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.objects.instance [None req-518e912f-17aa-4808-a3a1-ce1c561b9dfc tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lazy-loading 'fault' on Instance uuid 2718d3b6-c71c-4ae1-9544-0f8e4a43d989 {{(pid=121036) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:12:29.843285 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-518e912f-17aa-4808-a3a1-ce1c561b9dfc tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f5e4723258c64ce5885154dded2d3d43', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '287d111f08ac4b05bd5adec9c40d0620', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:29.851606 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-518e912f-17aa-4808-a3a1-ce1c561b9dfc tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989" status: 200 len: 1331 microversion: 2.1 time: 0.117138 Sep 04 01:12:29.866872 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 557/1084] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:29 2022] GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989 => generated 1331 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:12:30.067648 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-a5119d14-f7c3-41dc-9e3c-fd7b6c697c79 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:30.068887 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-a5119d14-f7c3-41dc-9e3c-fd7b6c697c79 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:30.075584 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a5119d14-f7c3-41dc-9e3c-fd7b6c697c79 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:30.076130 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a5119d14-f7c3-41dc-9e3c-fd7b6c697c79 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:30.076936 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a5119d14-f7c3-41dc-9e3c-fd7b6c697c79 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:30.077790 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a5119d14-f7c3-41dc-9e3c-fd7b6c697c79 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:30.079234 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a5119d14-f7c3-41dc-9e3c-fd7b6c697c79 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:30.079661 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a5119d14-f7c3-41dc-9e3c-fd7b6c697c79 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:30.233405 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-a5119d14-f7c3-41dc-9e3c-fd7b6c697c79 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:30.242366 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-a5119d14-f7c3-41dc-9e3c-fd7b6c697c79 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1562 microversion: 2.1 time: 0.180624 Sep 04 01:12:30.242944 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 528/1085] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:30 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1562 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:12:30.876212 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-026e2f7f-bcbd-4cb3-ac34-7d6279619d06 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:30.876828 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-026e2f7f-bcbd-4cb3-ac34-7d6279619d06 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 2718d3b6-c71c-4ae1-9544-0f8e4a43d989] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:30.883658 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-026e2f7f-bcbd-4cb3-ac34-7d6279619d06 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:30.884142 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-026e2f7f-bcbd-4cb3-ac34-7d6279619d06 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:30.884576 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-026e2f7f-bcbd-4cb3-ac34-7d6279619d06 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:30.885299 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-026e2f7f-bcbd-4cb3-ac34-7d6279619d06 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:30.885752 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-026e2f7f-bcbd-4cb3-ac34-7d6279619d06 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:30.886175 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-026e2f7f-bcbd-4cb3-ac34-7d6279619d06 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:30.900933 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.wsgi [None req-026e2f7f-bcbd-4cb3-ac34-7d6279619d06 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] HTTP exception thrown: Instance 2718d3b6-c71c-4ae1-9544-0f8e4a43d989 could not be found. Sep 04 01:12:30.902848 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-026e2f7f-bcbd-4cb3-ac34-7d6279619d06 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Request method failure captured: Sep 04 01:12:30.902848 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: request: GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989 HTTP/1.1 Sep 04 01:12:30.902848 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept: application/json Sep 04 01:12:30.902848 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept-Encoding: identity Sep 04 01:12:30.902848 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Connection: close Sep 04 01:12:30.902848 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Length: 0 Sep 04 01:12:30.902848 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Type: application/json Sep 04 01:12:30.902848 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Host: 10.222.0.42 Sep 04 01:12:30.902848 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Openstack-System-Scope: None Sep 04 01:12:30.902848 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: User-Agent: python-urllib3/1.26.12 Sep 04 01:12:30.902848 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Auth-Token: gAAAAABjE_nLdulKsCh7rP69EyY8StqBiRnQNFlCX5VoWZ9XzP_YTEis0NNuYGIELGjVYA3FFaOCC9PLYuhPddV3AgDhoAQw_S2o9MK0q1A57ehVT3gYS6mSkklOOLScx3a8r0TUFxeOMhMymWFJt03FSLp4K4nMUe8FttsVqGRsky_ik39T3Zg Sep 04 01:12:30.902848 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Id: None Sep 04 01:12:30.902848 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Name: None Sep 04 01:12:30.902848 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Identity-Status: Confirmed Sep 04 01:12:30.902848 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Is-Admin-Project: True Sep 04 01:12:30.902848 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Id: default Sep 04 01:12:30.902848 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Name: Default Sep 04 01:12:30.902848 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Id: 287d111f08ac4b05bd5adec9c40d0620 Sep 04 01:12:30.902848 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Name: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:12:30.902848 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Role: reader,member Sep 04 01:12:30.902848 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Roles: reader,member Sep 04 01:12:30.904572 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_287d111f08ac4b05bd5adec9c40d0620", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:12:30.904572 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:12:30.904572 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Id: 287d111f08ac4b05bd5adec9c40d0620 Sep 04 01:12:30.904572 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:12:30.904572 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User: tempest-AttachVolumeNegativeTest-825474236-project Sep 04 01:12:30.904572 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Id: default Sep 04 01:12:30.904572 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Name: Default Sep 04 01:12:30.904572 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Id: f5e4723258c64ce5885154dded2d3d43 Sep 04 01:12:30.904572 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Name: tempest-AttachVolumeNegativeTest-825474236-project Sep 04 01:12:30.904572 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: method: > Sep 04 01:12:30.904572 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: exception: Instance 2718d3b6-c71c-4ae1-9544-0f8e4a43d989 could not be found. Sep 04 01:12:30.904572 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: action_args: {'id': '2718d3b6-c71c-4ae1-9544-0f8e4a43d989'} Sep 04 01:12:30.906061 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:12:30.906061 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:30.906061 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:12:30.906061 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:12:30.906061 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:12:30.906061 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:12:30.906061 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:12:30.906061 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:12:30.906061 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:12:30.906061 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:12:30.906061 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2718d3b6-c71c-4ae1-9544-0f8e4a43d989 could not be found. Sep 04 01:12:30.906061 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:30.906061 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:30.906061 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:30.906061 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:30.906061 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:30.906061 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:30.906061 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:12:30.906061 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:12:30.906061 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:12:30.907803 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:12:30.907803 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 04 01:12:30.907803 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 04 01:12:30.907803 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:12:30.907803 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:12:30.907803 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:12:30.907803 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:12:30.907803 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2718d3b6-c71c-4ae1-9544-0f8e4a43d989 could not be found. Sep 04 01:12:30.907803 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:30.907803 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:30.907803 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:30.907803 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:30.907803 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:30.907803 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:30.907803 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:12:30.907803 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:12:30.907803 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2718d3b6-c71c-4ae1-9544-0f8e4a43d989 could not be found. Sep 04 01:12:30.907803 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:30.911820 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-026e2f7f-bcbd-4cb3-ac34-7d6279619d06 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Returning 404 to user: Instance 2718d3b6-c71c-4ae1-9544-0f8e4a43d989 could not be found. {{(pid=121036) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:12:30.911820 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-026e2f7f-bcbd-4cb3-ac34-7d6279619d06 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989" status: 404 len: 111 microversion: 2.1 time: 0.032402 Sep 04 01:12:30.911820 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 558/1086] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:30 2022] GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 04 01:12:30.913391 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-eceb6594-8a35-40d6-9238-b85e0cd6fa01 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '.version_select at 0x7fc10c6daa60>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:31.000732 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-eceb6594-8a35-40d6-9238-b85e0cd6fa01 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1092737247" status: 202 len: 0 microversion: 2.1 time: 0.090569 Sep 04 01:12:31.001331 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 529/1087] 10.222.0.42 () {58 vars in 1221 bytes} [Sun Sep 4 01:12:30 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1092737247 => generated 0 bytes in 93 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 04 01:12:33.058480 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-fde92315-1478-4e5b-8fdf-926a0f8bbf56 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:33.058984 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-fde92315-1478-4e5b-8fdf-926a0f8bbf56 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 2718d3b6-c71c-4ae1-9544-0f8e4a43d989] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:33.065727 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fde92315-1478-4e5b-8fdf-926a0f8bbf56 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:33.066210 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fde92315-1478-4e5b-8fdf-926a0f8bbf56 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:33.066657 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fde92315-1478-4e5b-8fdf-926a0f8bbf56 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:33.067474 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fde92315-1478-4e5b-8fdf-926a0f8bbf56 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:33.067930 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fde92315-1478-4e5b-8fdf-926a0f8bbf56 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:33.068354 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fde92315-1478-4e5b-8fdf-926a0f8bbf56 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:33.081872 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.wsgi [None req-fde92315-1478-4e5b-8fdf-926a0f8bbf56 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] HTTP exception thrown: Instance 2718d3b6-c71c-4ae1-9544-0f8e4a43d989 could not be found. Sep 04 01:12:33.083399 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-fde92315-1478-4e5b-8fdf-926a0f8bbf56 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Request method failure captured: Sep 04 01:12:33.083399 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: request: DELETE /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989 HTTP/1.1 Sep 04 01:12:33.083399 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept: application/json Sep 04 01:12:33.083399 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept-Encoding: identity Sep 04 01:12:33.083399 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Connection: close Sep 04 01:12:33.083399 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Length: 0 Sep 04 01:12:33.083399 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Type: application/json Sep 04 01:12:33.083399 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Host: 10.222.0.42 Sep 04 01:12:33.083399 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Openstack-System-Scope: None Sep 04 01:12:33.083399 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: User-Agent: python-urllib3/1.26.12 Sep 04 01:12:33.083399 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Auth-Token: gAAAAABjE_nLdulKsCh7rP69EyY8StqBiRnQNFlCX5VoWZ9XzP_YTEis0NNuYGIELGjVYA3FFaOCC9PLYuhPddV3AgDhoAQw_S2o9MK0q1A57ehVT3gYS6mSkklOOLScx3a8r0TUFxeOMhMymWFJt03FSLp4K4nMUe8FttsVqGRsky_ik39T3Zg Sep 04 01:12:33.083399 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Id: None Sep 04 01:12:33.083399 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Name: None Sep 04 01:12:33.083399 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Identity-Status: Confirmed Sep 04 01:12:33.083399 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Is-Admin-Project: True Sep 04 01:12:33.083399 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Id: default Sep 04 01:12:33.083399 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Name: Default Sep 04 01:12:33.083399 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Id: 287d111f08ac4b05bd5adec9c40d0620 Sep 04 01:12:33.083399 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Name: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:12:33.083399 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Role: reader,member Sep 04 01:12:33.083399 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Roles: reader,member Sep 04 01:12:33.084993 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_287d111f08ac4b05bd5adec9c40d0620", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:12:33.084993 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:12:33.084993 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Id: 287d111f08ac4b05bd5adec9c40d0620 Sep 04 01:12:33.084993 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:12:33.084993 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User: tempest-AttachVolumeNegativeTest-825474236-project Sep 04 01:12:33.084993 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Id: default Sep 04 01:12:33.084993 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Name: Default Sep 04 01:12:33.084993 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Id: f5e4723258c64ce5885154dded2d3d43 Sep 04 01:12:33.084993 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Name: tempest-AttachVolumeNegativeTest-825474236-project Sep 04 01:12:33.084993 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: method: > Sep 04 01:12:33.084993 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: exception: Instance 2718d3b6-c71c-4ae1-9544-0f8e4a43d989 could not be found. Sep 04 01:12:33.084993 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: action_args: {'id': '2718d3b6-c71c-4ae1-9544-0f8e4a43d989'} Sep 04 01:12:33.086179 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:12:33.086179 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:33.086179 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:12:33.086179 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:12:33.086179 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:12:33.086179 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:12:33.086179 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:12:33.086179 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:12:33.086179 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:12:33.086179 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:12:33.086179 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2718d3b6-c71c-4ae1-9544-0f8e4a43d989 could not be found. Sep 04 01:12:33.086179 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:33.086179 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:33.086179 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:33.086179 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:33.086179 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:33.086179 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:33.086179 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:12:33.086179 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:12:33.086179 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:12:33.087450 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:12:33.087450 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 04 01:12:33.087450 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 04 01:12:33.087450 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 04 01:12:33.087450 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 04 01:12:33.087450 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:12:33.087450 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:12:33.087450 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:12:33.087450 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:12:33.087450 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2718d3b6-c71c-4ae1-9544-0f8e4a43d989 could not be found. Sep 04 01:12:33.087450 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:33.087450 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:33.087450 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:33.087450 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:33.087450 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:33.087450 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:33.087450 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:12:33.087450 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:12:33.087450 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2718d3b6-c71c-4ae1-9544-0f8e4a43d989 could not be found. Sep 04 01:12:33.087450 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:33.088847 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-fde92315-1478-4e5b-8fdf-926a0f8bbf56 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Returning 404 to user: Instance 2718d3b6-c71c-4ae1-9544-0f8e4a43d989 could not be found. {{(pid=121036) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:12:33.088847 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-fde92315-1478-4e5b-8fdf-926a0f8bbf56 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "DELETE /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989" status: 404 len: 111 microversion: 2.1 time: 0.030427 Sep 04 01:12:33.088847 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 559/1088] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 4 01:12:33 2022] DELETE /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 04 01:12:33.093255 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-044b1ee4-7a0d-4376-8057-58404d77995e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:33.094521 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-044b1ee4-7a0d-4376-8057-58404d77995e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 2718d3b6-c71c-4ae1-9544-0f8e4a43d989] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:33.100566 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-044b1ee4-7a0d-4376-8057-58404d77995e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:33.101051 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-044b1ee4-7a0d-4376-8057-58404d77995e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:33.101592 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-044b1ee4-7a0d-4376-8057-58404d77995e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:33.102288 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-044b1ee4-7a0d-4376-8057-58404d77995e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:33.102745 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-044b1ee4-7a0d-4376-8057-58404d77995e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:33.103172 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-044b1ee4-7a0d-4376-8057-58404d77995e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:33.118059 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.wsgi [None req-044b1ee4-7a0d-4376-8057-58404d77995e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] HTTP exception thrown: Instance 2718d3b6-c71c-4ae1-9544-0f8e4a43d989 could not be found. Sep 04 01:12:33.119722 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-044b1ee4-7a0d-4376-8057-58404d77995e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Request method failure captured: Sep 04 01:12:33.119722 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: request: GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989 HTTP/1.1 Sep 04 01:12:33.119722 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept: application/json Sep 04 01:12:33.119722 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept-Encoding: identity Sep 04 01:12:33.119722 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Connection: close Sep 04 01:12:33.119722 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Length: 0 Sep 04 01:12:33.119722 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Type: application/json Sep 04 01:12:33.119722 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Host: 10.222.0.42 Sep 04 01:12:33.119722 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Openstack-System-Scope: None Sep 04 01:12:33.119722 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: User-Agent: python-urllib3/1.26.12 Sep 04 01:12:33.119722 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Auth-Token: gAAAAABjE_nLdulKsCh7rP69EyY8StqBiRnQNFlCX5VoWZ9XzP_YTEis0NNuYGIELGjVYA3FFaOCC9PLYuhPddV3AgDhoAQw_S2o9MK0q1A57ehVT3gYS6mSkklOOLScx3a8r0TUFxeOMhMymWFJt03FSLp4K4nMUe8FttsVqGRsky_ik39T3Zg Sep 04 01:12:33.119722 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Id: None Sep 04 01:12:33.119722 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Name: None Sep 04 01:12:33.119722 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Identity-Status: Confirmed Sep 04 01:12:33.119722 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Is-Admin-Project: True Sep 04 01:12:33.119722 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Id: default Sep 04 01:12:33.119722 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Name: Default Sep 04 01:12:33.119722 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Id: 287d111f08ac4b05bd5adec9c40d0620 Sep 04 01:12:33.119722 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Name: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:12:33.119722 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Role: reader,member Sep 04 01:12:33.119722 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Roles: reader,member Sep 04 01:12:33.121399 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_287d111f08ac4b05bd5adec9c40d0620", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:12:33.121399 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:12:33.121399 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Id: 287d111f08ac4b05bd5adec9c40d0620 Sep 04 01:12:33.121399 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:12:33.121399 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User: tempest-AttachVolumeNegativeTest-825474236-project Sep 04 01:12:33.121399 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Id: default Sep 04 01:12:33.121399 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Name: Default Sep 04 01:12:33.121399 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Id: f5e4723258c64ce5885154dded2d3d43 Sep 04 01:12:33.121399 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Name: tempest-AttachVolumeNegativeTest-825474236-project Sep 04 01:12:33.121399 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: method: > Sep 04 01:12:33.121399 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: exception: Instance 2718d3b6-c71c-4ae1-9544-0f8e4a43d989 could not be found. Sep 04 01:12:33.121399 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: action_args: {'id': '2718d3b6-c71c-4ae1-9544-0f8e4a43d989'} Sep 04 01:12:33.125712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:12:33.125712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:33.125712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:12:33.125712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:12:33.125712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:12:33.125712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:12:33.125712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:12:33.125712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:12:33.125712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:12:33.125712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:12:33.125712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2718d3b6-c71c-4ae1-9544-0f8e4a43d989 could not be found. Sep 04 01:12:33.125712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:33.125712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:33.125712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:33.125712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:33.125712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:33.125712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:33.125712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:12:33.125712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:12:33.125712 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:12:33.127563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:12:33.127563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 04 01:12:33.127563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 04 01:12:33.127563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:12:33.127563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:12:33.127563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:12:33.127563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:12:33.127563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2718d3b6-c71c-4ae1-9544-0f8e4a43d989 could not be found. Sep 04 01:12:33.127563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:33.127563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:33.127563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:33.127563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:33.127563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:33.127563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:33.127563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:12:33.127563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:12:33.127563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2718d3b6-c71c-4ae1-9544-0f8e4a43d989 could not be found. Sep 04 01:12:33.127563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:33.128876 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-044b1ee4-7a0d-4376-8057-58404d77995e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Returning 404 to user: Instance 2718d3b6-c71c-4ae1-9544-0f8e4a43d989 could not be found. {{(pid=121037) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:12:33.128876 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-044b1ee4-7a0d-4376-8057-58404d77995e tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989" status: 404 len: 111 microversion: 2.1 time: 0.031773 Sep 04 01:12:33.128876 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 530/1089] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:33 2022] GET /compute/v2.1/servers/2718d3b6-c71c-4ae1-9544-0f8e4a43d989 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 04 01:12:34.459134 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-6a89d268-8aac-4d96-8d3c-4fabac8c38e7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:34.459679 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-6a89d268-8aac-4d96-8d3c-4fabac8c38e7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 16b0498f-4cb8-491a-92cf-dac9890c7b4a] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:34.466037 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6a89d268-8aac-4d96-8d3c-4fabac8c38e7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:34.466508 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6a89d268-8aac-4d96-8d3c-4fabac8c38e7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:34.466935 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6a89d268-8aac-4d96-8d3c-4fabac8c38e7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:34.467616 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6a89d268-8aac-4d96-8d3c-4fabac8c38e7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:34.468060 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6a89d268-8aac-4d96-8d3c-4fabac8c38e7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:34.468476 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6a89d268-8aac-4d96-8d3c-4fabac8c38e7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:34.482005 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.wsgi [None req-6a89d268-8aac-4d96-8d3c-4fabac8c38e7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] HTTP exception thrown: Instance 16b0498f-4cb8-491a-92cf-dac9890c7b4a could not be found. Sep 04 01:12:34.483271 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-6a89d268-8aac-4d96-8d3c-4fabac8c38e7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Request method failure captured: Sep 04 01:12:34.483271 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: request: DELETE /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a HTTP/1.1 Sep 04 01:12:34.483271 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept: application/json Sep 04 01:12:34.483271 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept-Encoding: identity Sep 04 01:12:34.483271 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Connection: close Sep 04 01:12:34.483271 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Length: 0 Sep 04 01:12:34.483271 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Type: application/json Sep 04 01:12:34.483271 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Host: 10.222.0.42 Sep 04 01:12:34.483271 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Openstack-System-Scope: None Sep 04 01:12:34.483271 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: User-Agent: python-urllib3/1.26.12 Sep 04 01:12:34.483271 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Auth-Token: gAAAAABjE_nLdulKsCh7rP69EyY8StqBiRnQNFlCX5VoWZ9XzP_YTEis0NNuYGIELGjVYA3FFaOCC9PLYuhPddV3AgDhoAQw_S2o9MK0q1A57ehVT3gYS6mSkklOOLScx3a8r0TUFxeOMhMymWFJt03FSLp4K4nMUe8FttsVqGRsky_ik39T3Zg Sep 04 01:12:34.483271 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Id: None Sep 04 01:12:34.483271 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Name: None Sep 04 01:12:34.483271 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Identity-Status: Confirmed Sep 04 01:12:34.483271 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Is-Admin-Project: True Sep 04 01:12:34.483271 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Id: default Sep 04 01:12:34.483271 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Name: Default Sep 04 01:12:34.483271 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Id: 287d111f08ac4b05bd5adec9c40d0620 Sep 04 01:12:34.483271 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Name: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:12:34.483271 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Role: reader,member Sep 04 01:12:34.483271 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Roles: reader,member Sep 04 01:12:34.484544 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_287d111f08ac4b05bd5adec9c40d0620", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:12:34.484544 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:12:34.484544 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Id: 287d111f08ac4b05bd5adec9c40d0620 Sep 04 01:12:34.484544 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:12:34.484544 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User: tempest-AttachVolumeNegativeTest-825474236-project Sep 04 01:12:34.484544 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Id: default Sep 04 01:12:34.484544 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Name: Default Sep 04 01:12:34.484544 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Id: f5e4723258c64ce5885154dded2d3d43 Sep 04 01:12:34.484544 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Name: tempest-AttachVolumeNegativeTest-825474236-project Sep 04 01:12:34.484544 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: method: > Sep 04 01:12:34.484544 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: exception: Instance 16b0498f-4cb8-491a-92cf-dac9890c7b4a could not be found. Sep 04 01:12:34.484544 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: action_args: {'id': '16b0498f-4cb8-491a-92cf-dac9890c7b4a'} Sep 04 01:12:34.485569 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:12:34.485569 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:34.485569 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:12:34.485569 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:12:34.485569 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:12:34.485569 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:12:34.485569 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:12:34.485569 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:12:34.485569 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:12:34.485569 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:12:34.485569 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 16b0498f-4cb8-491a-92cf-dac9890c7b4a could not be found. Sep 04 01:12:34.485569 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:34.485569 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:34.485569 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:34.485569 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:34.485569 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:34.485569 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:34.485569 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:12:34.485569 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:12:34.485569 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:12:34.486789 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:12:34.486789 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 04 01:12:34.486789 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 04 01:12:34.486789 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 04 01:12:34.486789 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 04 01:12:34.486789 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:12:34.486789 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:12:34.486789 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:12:34.486789 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:12:34.486789 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 16b0498f-4cb8-491a-92cf-dac9890c7b4a could not be found. Sep 04 01:12:34.486789 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:34.486789 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:34.486789 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:34.486789 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:34.486789 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:34.486789 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:34.486789 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:12:34.486789 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:12:34.486789 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 16b0498f-4cb8-491a-92cf-dac9890c7b4a could not be found. Sep 04 01:12:34.486789 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:34.488044 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-6a89d268-8aac-4d96-8d3c-4fabac8c38e7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Returning 404 to user: Instance 16b0498f-4cb8-491a-92cf-dac9890c7b4a could not be found. {{(pid=121036) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:12:34.488044 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-6a89d268-8aac-4d96-8d3c-4fabac8c38e7 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "DELETE /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a" status: 404 len: 111 microversion: 2.1 time: 0.030063 Sep 04 01:12:34.488044 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 560/1090] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 4 01:12:34 2022] DELETE /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 04 01:12:34.493610 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-614b1fa0-b978-4e9d-88a0-6fe2feb843e5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:34.494353 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-614b1fa0-b978-4e9d-88a0-6fe2feb843e5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 16b0498f-4cb8-491a-92cf-dac9890c7b4a] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:34.500782 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-614b1fa0-b978-4e9d-88a0-6fe2feb843e5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:34.501818 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-614b1fa0-b978-4e9d-88a0-6fe2feb843e5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:34.502260 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-614b1fa0-b978-4e9d-88a0-6fe2feb843e5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:34.502939 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-614b1fa0-b978-4e9d-88a0-6fe2feb843e5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:34.503388 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-614b1fa0-b978-4e9d-88a0-6fe2feb843e5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:34.503811 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-614b1fa0-b978-4e9d-88a0-6fe2feb843e5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:34.517705 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.wsgi [None req-614b1fa0-b978-4e9d-88a0-6fe2feb843e5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] HTTP exception thrown: Instance 16b0498f-4cb8-491a-92cf-dac9890c7b4a could not be found. Sep 04 01:12:34.519216 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-614b1fa0-b978-4e9d-88a0-6fe2feb843e5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Request method failure captured: Sep 04 01:12:34.519216 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: request: GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a HTTP/1.1 Sep 04 01:12:34.519216 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept: application/json Sep 04 01:12:34.519216 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept-Encoding: identity Sep 04 01:12:34.519216 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Connection: close Sep 04 01:12:34.519216 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Length: 0 Sep 04 01:12:34.519216 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Type: application/json Sep 04 01:12:34.519216 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Host: 10.222.0.42 Sep 04 01:12:34.519216 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Openstack-System-Scope: None Sep 04 01:12:34.519216 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: User-Agent: python-urllib3/1.26.12 Sep 04 01:12:34.519216 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Auth-Token: gAAAAABjE_nLdulKsCh7rP69EyY8StqBiRnQNFlCX5VoWZ9XzP_YTEis0NNuYGIELGjVYA3FFaOCC9PLYuhPddV3AgDhoAQw_S2o9MK0q1A57ehVT3gYS6mSkklOOLScx3a8r0TUFxeOMhMymWFJt03FSLp4K4nMUe8FttsVqGRsky_ik39T3Zg Sep 04 01:12:34.519216 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Id: None Sep 04 01:12:34.519216 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Name: None Sep 04 01:12:34.519216 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Identity-Status: Confirmed Sep 04 01:12:34.519216 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Is-Admin-Project: True Sep 04 01:12:34.519216 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Id: default Sep 04 01:12:34.519216 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Name: Default Sep 04 01:12:34.519216 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Id: 287d111f08ac4b05bd5adec9c40d0620 Sep 04 01:12:34.519216 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Name: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:12:34.519216 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Role: reader,member Sep 04 01:12:34.519216 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Roles: reader,member Sep 04 01:12:34.520831 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_287d111f08ac4b05bd5adec9c40d0620", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:12:34.520831 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:12:34.520831 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Id: 287d111f08ac4b05bd5adec9c40d0620 Sep 04 01:12:34.520831 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:12:34.520831 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User: tempest-AttachVolumeNegativeTest-825474236-project Sep 04 01:12:34.520831 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Id: default Sep 04 01:12:34.520831 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Name: Default Sep 04 01:12:34.520831 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Id: f5e4723258c64ce5885154dded2d3d43 Sep 04 01:12:34.520831 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Name: tempest-AttachVolumeNegativeTest-825474236-project Sep 04 01:12:34.520831 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: method: > Sep 04 01:12:34.520831 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: exception: Instance 16b0498f-4cb8-491a-92cf-dac9890c7b4a could not be found. Sep 04 01:12:34.520831 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: action_args: {'id': '16b0498f-4cb8-491a-92cf-dac9890c7b4a'} Sep 04 01:12:34.521821 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:12:34.521821 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:34.521821 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:12:34.521821 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:12:34.521821 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:12:34.521821 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:12:34.521821 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:12:34.521821 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:12:34.521821 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:12:34.521821 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:12:34.521821 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 16b0498f-4cb8-491a-92cf-dac9890c7b4a could not be found. Sep 04 01:12:34.521821 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:34.521821 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:34.521821 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:34.521821 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:34.521821 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:34.521821 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:34.521821 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:12:34.521821 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:12:34.521821 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:12:34.523263 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:12:34.523263 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 04 01:12:34.523263 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 04 01:12:34.523263 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:12:34.523263 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:12:34.523263 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:12:34.523263 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:12:34.523263 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 16b0498f-4cb8-491a-92cf-dac9890c7b4a could not be found. Sep 04 01:12:34.523263 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:34.523263 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:34.523263 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:34.523263 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:34.523263 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:34.523263 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:34.523263 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:12:34.523263 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:12:34.523263 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 16b0498f-4cb8-491a-92cf-dac9890c7b4a could not be found. Sep 04 01:12:34.523263 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:34.524515 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-614b1fa0-b978-4e9d-88a0-6fe2feb843e5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Returning 404 to user: Instance 16b0498f-4cb8-491a-92cf-dac9890c7b4a could not be found. {{(pid=121037) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:12:34.524515 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-614b1fa0-b978-4e9d-88a0-6fe2feb843e5 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a" status: 404 len: 111 microversion: 2.1 time: 0.031188 Sep 04 01:12:34.524515 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 531/1091] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:34 2022] GET /compute/v2.1/servers/16b0498f-4cb8-491a-92cf-dac9890c7b4a => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 04 01:12:34.529125 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-5fc2eb21-314a-4924-89d8-63b0625f1068 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:34.529624 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-5fc2eb21-314a-4924-89d8-63b0625f1068 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:34.535728 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5fc2eb21-314a-4924-89d8-63b0625f1068 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:34.536199 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5fc2eb21-314a-4924-89d8-63b0625f1068 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:34.536628 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5fc2eb21-314a-4924-89d8-63b0625f1068 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:34.537330 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5fc2eb21-314a-4924-89d8-63b0625f1068 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:34.537779 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5fc2eb21-314a-4924-89d8-63b0625f1068 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:34.538198 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5fc2eb21-314a-4924-89d8-63b0625f1068 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:34.551329 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.wsgi [None req-5fc2eb21-314a-4924-89d8-63b0625f1068 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] HTTP exception thrown: Instance 205e4f5b-43ef-494a-a6ad-64f7445e6b59 could not be found. Sep 04 01:12:34.552868 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-5fc2eb21-314a-4924-89d8-63b0625f1068 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Request method failure captured: Sep 04 01:12:34.552868 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: request: DELETE /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 HTTP/1.1 Sep 04 01:12:34.552868 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept: application/json Sep 04 01:12:34.552868 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept-Encoding: identity Sep 04 01:12:34.552868 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Connection: close Sep 04 01:12:34.552868 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Length: 0 Sep 04 01:12:34.552868 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Type: application/json Sep 04 01:12:34.552868 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Host: 10.222.0.42 Sep 04 01:12:34.552868 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Openstack-System-Scope: None Sep 04 01:12:34.552868 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: User-Agent: python-urllib3/1.26.12 Sep 04 01:12:34.552868 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Auth-Token: gAAAAABjE_nLdulKsCh7rP69EyY8StqBiRnQNFlCX5VoWZ9XzP_YTEis0NNuYGIELGjVYA3FFaOCC9PLYuhPddV3AgDhoAQw_S2o9MK0q1A57ehVT3gYS6mSkklOOLScx3a8r0TUFxeOMhMymWFJt03FSLp4K4nMUe8FttsVqGRsky_ik39T3Zg Sep 04 01:12:34.552868 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Id: None Sep 04 01:12:34.552868 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Name: None Sep 04 01:12:34.552868 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Identity-Status: Confirmed Sep 04 01:12:34.552868 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Is-Admin-Project: True Sep 04 01:12:34.552868 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Id: default Sep 04 01:12:34.552868 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Name: Default Sep 04 01:12:34.552868 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Id: 287d111f08ac4b05bd5adec9c40d0620 Sep 04 01:12:34.552868 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Name: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:12:34.552868 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Role: reader,member Sep 04 01:12:34.552868 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Roles: reader,member Sep 04 01:12:34.554415 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_287d111f08ac4b05bd5adec9c40d0620", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:12:34.554415 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:12:34.554415 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Id: 287d111f08ac4b05bd5adec9c40d0620 Sep 04 01:12:34.554415 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:12:34.554415 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User: tempest-AttachVolumeNegativeTest-825474236-project Sep 04 01:12:34.554415 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Id: default Sep 04 01:12:34.554415 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Name: Default Sep 04 01:12:34.554415 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Id: f5e4723258c64ce5885154dded2d3d43 Sep 04 01:12:34.554415 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Name: tempest-AttachVolumeNegativeTest-825474236-project Sep 04 01:12:34.554415 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: method: > Sep 04 01:12:34.554415 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: exception: Instance 205e4f5b-43ef-494a-a6ad-64f7445e6b59 could not be found. Sep 04 01:12:34.554415 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: action_args: {'id': '205e4f5b-43ef-494a-a6ad-64f7445e6b59'} Sep 04 01:12:34.555242 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:12:34.555242 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:34.555242 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:12:34.555242 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:12:34.555242 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:12:34.555242 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:12:34.555242 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:12:34.555242 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:12:34.555242 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:12:34.555242 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:12:34.555242 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 205e4f5b-43ef-494a-a6ad-64f7445e6b59 could not be found. Sep 04 01:12:34.555242 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:34.555242 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:34.555242 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:34.555242 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:34.555242 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:34.555242 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:34.555242 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:12:34.555242 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:12:34.555242 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:12:34.556677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:12:34.556677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 04 01:12:34.556677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 04 01:12:34.556677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 04 01:12:34.556677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 04 01:12:34.556677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:12:34.556677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:12:34.556677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:12:34.556677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:12:34.556677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 205e4f5b-43ef-494a-a6ad-64f7445e6b59 could not be found. Sep 04 01:12:34.556677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:34.556677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:34.556677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:34.556677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:34.556677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:34.556677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:34.556677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:12:34.556677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:12:34.556677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 205e4f5b-43ef-494a-a6ad-64f7445e6b59 could not be found. Sep 04 01:12:34.556677 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:34.558195 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-5fc2eb21-314a-4924-89d8-63b0625f1068 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Returning 404 to user: Instance 205e4f5b-43ef-494a-a6ad-64f7445e6b59 could not be found. {{(pid=121036) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:12:34.558195 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-5fc2eb21-314a-4924-89d8-63b0625f1068 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "DELETE /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 404 len: 111 microversion: 2.1 time: 0.028993 Sep 04 01:12:34.558195 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 561/1092] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 4 01:12:34 2022] DELETE /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 04 01:12:34.562458 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-1a861175-3951-4d3e-b206-3927d46e4329 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:34.563185 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-1a861175-3951-4d3e-b206-3927d46e4329 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 205e4f5b-43ef-494a-a6ad-64f7445e6b59] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:34.569721 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1a861175-3951-4d3e-b206-3927d46e4329 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:34.570195 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1a861175-3951-4d3e-b206-3927d46e4329 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:34.570627 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1a861175-3951-4d3e-b206-3927d46e4329 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:34.571307 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1a861175-3951-4d3e-b206-3927d46e4329 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:34.571760 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1a861175-3951-4d3e-b206-3927d46e4329 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:34.572181 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1a861175-3951-4d3e-b206-3927d46e4329 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:34.585885 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.wsgi [None req-1a861175-3951-4d3e-b206-3927d46e4329 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] HTTP exception thrown: Instance 205e4f5b-43ef-494a-a6ad-64f7445e6b59 could not be found. Sep 04 01:12:34.587374 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-1a861175-3951-4d3e-b206-3927d46e4329 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Request method failure captured: Sep 04 01:12:34.587374 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: request: GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 HTTP/1.1 Sep 04 01:12:34.587374 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept: application/json Sep 04 01:12:34.587374 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept-Encoding: identity Sep 04 01:12:34.587374 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Connection: close Sep 04 01:12:34.587374 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Length: 0 Sep 04 01:12:34.587374 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Type: application/json Sep 04 01:12:34.587374 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Host: 10.222.0.42 Sep 04 01:12:34.587374 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Openstack-System-Scope: None Sep 04 01:12:34.587374 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: User-Agent: python-urllib3/1.26.12 Sep 04 01:12:34.587374 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Auth-Token: gAAAAABjE_nLdulKsCh7rP69EyY8StqBiRnQNFlCX5VoWZ9XzP_YTEis0NNuYGIELGjVYA3FFaOCC9PLYuhPddV3AgDhoAQw_S2o9MK0q1A57ehVT3gYS6mSkklOOLScx3a8r0TUFxeOMhMymWFJt03FSLp4K4nMUe8FttsVqGRsky_ik39T3Zg Sep 04 01:12:34.587374 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Id: None Sep 04 01:12:34.587374 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Name: None Sep 04 01:12:34.587374 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Identity-Status: Confirmed Sep 04 01:12:34.587374 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Is-Admin-Project: True Sep 04 01:12:34.587374 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Id: default Sep 04 01:12:34.587374 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Name: Default Sep 04 01:12:34.587374 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Id: 287d111f08ac4b05bd5adec9c40d0620 Sep 04 01:12:34.587374 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Name: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:12:34.587374 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Role: reader,member Sep 04 01:12:34.587374 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Roles: reader,member Sep 04 01:12:34.589040 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_287d111f08ac4b05bd5adec9c40d0620", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:12:34.589040 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:12:34.589040 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Id: 287d111f08ac4b05bd5adec9c40d0620 Sep 04 01:12:34.589040 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:12:34.589040 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User: tempest-AttachVolumeNegativeTest-825474236-project Sep 04 01:12:34.589040 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Id: default Sep 04 01:12:34.589040 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Name: Default Sep 04 01:12:34.589040 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Id: f5e4723258c64ce5885154dded2d3d43 Sep 04 01:12:34.589040 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Name: tempest-AttachVolumeNegativeTest-825474236-project Sep 04 01:12:34.589040 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: method: > Sep 04 01:12:34.589040 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: exception: Instance 205e4f5b-43ef-494a-a6ad-64f7445e6b59 could not be found. Sep 04 01:12:34.589040 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: action_args: {'id': '205e4f5b-43ef-494a-a6ad-64f7445e6b59'} Sep 04 01:12:34.589947 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:12:34.589947 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:34.589947 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:12:34.589947 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:12:34.589947 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:12:34.589947 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:12:34.589947 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:12:34.589947 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:12:34.589947 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:12:34.589947 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:12:34.589947 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 205e4f5b-43ef-494a-a6ad-64f7445e6b59 could not be found. Sep 04 01:12:34.589947 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:34.589947 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:34.589947 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:34.589947 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:34.589947 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:34.589947 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:34.589947 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:12:34.589947 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:12:34.589947 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:12:34.591381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:12:34.591381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 04 01:12:34.591381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 04 01:12:34.591381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:12:34.591381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:12:34.591381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:12:34.591381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:12:34.591381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 205e4f5b-43ef-494a-a6ad-64f7445e6b59 could not be found. Sep 04 01:12:34.591381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:34.591381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:34.591381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:34.591381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:34.591381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:34.591381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:34.591381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:12:34.591381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:12:34.591381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 205e4f5b-43ef-494a-a6ad-64f7445e6b59 could not be found. Sep 04 01:12:34.591381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:34.592709 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-1a861175-3951-4d3e-b206-3927d46e4329 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Returning 404 to user: Instance 205e4f5b-43ef-494a-a6ad-64f7445e6b59 could not be found. {{(pid=121037) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:12:34.592709 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-1a861175-3951-4d3e-b206-3927d46e4329 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59" status: 404 len: 111 microversion: 2.1 time: 0.030004 Sep 04 01:12:34.592709 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 532/1093] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:34 2022] GET /compute/v2.1/servers/205e4f5b-43ef-494a-a6ad-64f7445e6b59 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 04 01:12:35.871853 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-82e00cc0-2736-481a-8db1-044266c24a6b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:35.872358 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-82e00cc0-2736-481a-8db1-044266c24a6b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:35.879427 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-82e00cc0-2736-481a-8db1-044266c24a6b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:35.879912 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-82e00cc0-2736-481a-8db1-044266c24a6b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:35.880352 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-82e00cc0-2736-481a-8db1-044266c24a6b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:35.881061 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-82e00cc0-2736-481a-8db1-044266c24a6b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:35.881537 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-82e00cc0-2736-481a-8db1-044266c24a6b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:35.881965 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-82e00cc0-2736-481a-8db1-044266c24a6b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:35.898058 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.wsgi [None req-82e00cc0-2736-481a-8db1-044266c24a6b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] HTTP exception thrown: Instance 784b2b9b-9890-4f9e-be75-2548b520e138 could not be found. Sep 04 01:12:35.899620 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-82e00cc0-2736-481a-8db1-044266c24a6b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Request method failure captured: Sep 04 01:12:35.899620 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: request: DELETE /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 HTTP/1.1 Sep 04 01:12:35.899620 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept: application/json Sep 04 01:12:35.899620 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept-Encoding: identity Sep 04 01:12:35.899620 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Connection: close Sep 04 01:12:35.899620 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Length: 0 Sep 04 01:12:35.899620 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Type: application/json Sep 04 01:12:35.899620 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Host: 10.222.0.42 Sep 04 01:12:35.899620 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Openstack-System-Scope: None Sep 04 01:12:35.899620 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: User-Agent: python-urllib3/1.26.12 Sep 04 01:12:35.899620 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Auth-Token: gAAAAABjE_nLdulKsCh7rP69EyY8StqBiRnQNFlCX5VoWZ9XzP_YTEis0NNuYGIELGjVYA3FFaOCC9PLYuhPddV3AgDhoAQw_S2o9MK0q1A57ehVT3gYS6mSkklOOLScx3a8r0TUFxeOMhMymWFJt03FSLp4K4nMUe8FttsVqGRsky_ik39T3Zg Sep 04 01:12:35.899620 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Id: None Sep 04 01:12:35.899620 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Name: None Sep 04 01:12:35.899620 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Identity-Status: Confirmed Sep 04 01:12:35.899620 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Is-Admin-Project: True Sep 04 01:12:35.899620 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Id: default Sep 04 01:12:35.899620 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Name: Default Sep 04 01:12:35.899620 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Id: 287d111f08ac4b05bd5adec9c40d0620 Sep 04 01:12:35.899620 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Name: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:12:35.899620 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Role: reader,member Sep 04 01:12:35.899620 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Roles: reader,member Sep 04 01:12:35.901206 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_287d111f08ac4b05bd5adec9c40d0620", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:12:35.901206 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:12:35.901206 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Id: 287d111f08ac4b05bd5adec9c40d0620 Sep 04 01:12:35.901206 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:12:35.901206 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User: tempest-AttachVolumeNegativeTest-825474236-project Sep 04 01:12:35.901206 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Id: default Sep 04 01:12:35.901206 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Name: Default Sep 04 01:12:35.901206 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Id: f5e4723258c64ce5885154dded2d3d43 Sep 04 01:12:35.901206 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Name: tempest-AttachVolumeNegativeTest-825474236-project Sep 04 01:12:35.901206 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: method: > Sep 04 01:12:35.901206 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: exception: Instance 784b2b9b-9890-4f9e-be75-2548b520e138 could not be found. Sep 04 01:12:35.901206 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: action_args: {'id': '784b2b9b-9890-4f9e-be75-2548b520e138'} Sep 04 01:12:35.902113 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:12:35.902113 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:35.902113 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:12:35.902113 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:12:35.902113 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:12:35.902113 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:12:35.902113 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:12:35.902113 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:12:35.902113 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:12:35.902113 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:12:35.902113 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 784b2b9b-9890-4f9e-be75-2548b520e138 could not be found. Sep 04 01:12:35.902113 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:35.902113 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:35.902113 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:35.902113 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:35.902113 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:35.902113 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:35.902113 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:12:35.902113 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:12:35.902113 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:12:35.903686 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:12:35.903686 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 04 01:12:35.903686 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 04 01:12:35.903686 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 04 01:12:35.903686 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 04 01:12:35.903686 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:12:35.903686 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:12:35.903686 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:12:35.903686 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:12:35.903686 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 784b2b9b-9890-4f9e-be75-2548b520e138 could not be found. Sep 04 01:12:35.903686 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:35.903686 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:35.903686 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:35.903686 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:35.903686 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:35.903686 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:35.903686 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:12:35.903686 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:12:35.903686 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 784b2b9b-9890-4f9e-be75-2548b520e138 could not be found. Sep 04 01:12:35.903686 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:35.905301 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-82e00cc0-2736-481a-8db1-044266c24a6b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Returning 404 to user: Instance 784b2b9b-9890-4f9e-be75-2548b520e138 could not be found. {{(pid=121036) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:12:35.905301 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-82e00cc0-2736-481a-8db1-044266c24a6b tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "DELETE /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 404 len: 111 microversion: 2.1 time: 0.033058 Sep 04 01:12:35.905301 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 562/1094] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 4 01:12:35 2022] DELETE /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 04 01:12:35.910551 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-32fc4b45-b1a3-4e41-a60d-1b9e907daa10 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:35.911163 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-32fc4b45-b1a3-4e41-a60d-1b9e907daa10 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] [instance: 784b2b9b-9890-4f9e-be75-2548b520e138] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:35.917885 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-32fc4b45-b1a3-4e41-a60d-1b9e907daa10 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:35.918388 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-32fc4b45-b1a3-4e41-a60d-1b9e907daa10 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:35.918820 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-32fc4b45-b1a3-4e41-a60d-1b9e907daa10 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:35.919503 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-32fc4b45-b1a3-4e41-a60d-1b9e907daa10 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:35.919950 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-32fc4b45-b1a3-4e41-a60d-1b9e907daa10 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:35.920372 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-32fc4b45-b1a3-4e41-a60d-1b9e907daa10 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:35.933916 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.wsgi [None req-32fc4b45-b1a3-4e41-a60d-1b9e907daa10 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] HTTP exception thrown: Instance 784b2b9b-9890-4f9e-be75-2548b520e138 could not be found. Sep 04 01:12:35.935421 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-32fc4b45-b1a3-4e41-a60d-1b9e907daa10 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Request method failure captured: Sep 04 01:12:35.935421 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: request: GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 HTTP/1.1 Sep 04 01:12:35.935421 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept: application/json Sep 04 01:12:35.935421 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept-Encoding: identity Sep 04 01:12:35.935421 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Connection: close Sep 04 01:12:35.935421 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Length: 0 Sep 04 01:12:35.935421 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Type: application/json Sep 04 01:12:35.935421 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Host: 10.222.0.42 Sep 04 01:12:35.935421 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Openstack-System-Scope: None Sep 04 01:12:35.935421 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: User-Agent: python-urllib3/1.26.12 Sep 04 01:12:35.935421 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Auth-Token: gAAAAABjE_nLdulKsCh7rP69EyY8StqBiRnQNFlCX5VoWZ9XzP_YTEis0NNuYGIELGjVYA3FFaOCC9PLYuhPddV3AgDhoAQw_S2o9MK0q1A57ehVT3gYS6mSkklOOLScx3a8r0TUFxeOMhMymWFJt03FSLp4K4nMUe8FttsVqGRsky_ik39T3Zg Sep 04 01:12:35.935421 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Id: None Sep 04 01:12:35.935421 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Name: None Sep 04 01:12:35.935421 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Identity-Status: Confirmed Sep 04 01:12:35.935421 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Is-Admin-Project: True Sep 04 01:12:35.935421 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Id: default Sep 04 01:12:35.935421 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Name: Default Sep 04 01:12:35.935421 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Id: 287d111f08ac4b05bd5adec9c40d0620 Sep 04 01:12:35.935421 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Name: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:12:35.935421 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Role: reader,member Sep 04 01:12:35.935421 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Roles: reader,member Sep 04 01:12:35.936970 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_287d111f08ac4b05bd5adec9c40d0620", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/287d111f08ac4b05bd5adec9c40d0620"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:12:35.936970 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:12:35.936970 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Id: 287d111f08ac4b05bd5adec9c40d0620 Sep 04 01:12:35.936970 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-825474236 Sep 04 01:12:35.936970 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User: tempest-AttachVolumeNegativeTest-825474236-project Sep 04 01:12:35.936970 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Id: default Sep 04 01:12:35.936970 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Name: Default Sep 04 01:12:35.936970 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Id: f5e4723258c64ce5885154dded2d3d43 Sep 04 01:12:35.936970 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Name: tempest-AttachVolumeNegativeTest-825474236-project Sep 04 01:12:35.936970 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: method: > Sep 04 01:12:35.936970 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: exception: Instance 784b2b9b-9890-4f9e-be75-2548b520e138 could not be found. Sep 04 01:12:35.936970 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: action_args: {'id': '784b2b9b-9890-4f9e-be75-2548b520e138'} Sep 04 01:12:35.937865 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:12:35.937865 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:35.937865 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:12:35.937865 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:12:35.937865 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:12:35.937865 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:12:35.937865 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:12:35.937865 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:12:35.937865 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:12:35.937865 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:12:35.937865 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 784b2b9b-9890-4f9e-be75-2548b520e138 could not be found. Sep 04 01:12:35.937865 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:35.937865 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:35.937865 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:35.937865 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:35.937865 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:35.937865 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:35.937865 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:12:35.937865 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:12:35.937865 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:12:35.939843 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:12:35.939843 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 04 01:12:35.939843 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 04 01:12:35.939843 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:12:35.939843 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:12:35.939843 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:12:35.939843 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:12:35.939843 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 784b2b9b-9890-4f9e-be75-2548b520e138 could not be found. Sep 04 01:12:35.939843 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:35.939843 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:35.939843 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:35.939843 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:35.939843 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:35.939843 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:35.939843 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:12:35.939843 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:12:35.939843 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 784b2b9b-9890-4f9e-be75-2548b520e138 could not be found. Sep 04 01:12:35.939843 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:35.940920 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-32fc4b45-b1a3-4e41-a60d-1b9e907daa10 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] Returning 404 to user: Instance 784b2b9b-9890-4f9e-be75-2548b520e138 could not be found. {{(pid=121037) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:12:35.940920 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-32fc4b45-b1a3-4e41-a60d-1b9e907daa10 tempest-AttachVolumeNegativeTest-825474236 tempest-AttachVolumeNegativeTest-825474236-project] 10.222.0.42 "GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138" status: 404 len: 111 microversion: 2.1 time: 0.030144 Sep 04 01:12:35.940920 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 533/1095] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:12:35 2022] GET /compute/v2.1/servers/784b2b9b-9890-4f9e-be75-2548b520e138 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 04 01:12:39.271238 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-36d9f0be-a13f-4e7d-bb08-8460757f6314 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:39.277764 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-36d9f0be-a13f-4e7d-bb08-8460757f6314 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:39.280230 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-36d9f0be-a13f-4e7d-bb08-8460757f6314 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:39.280823 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-36d9f0be-a13f-4e7d-bb08-8460757f6314 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:39.281375 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-36d9f0be-a13f-4e7d-bb08-8460757f6314 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:39.282217 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-36d9f0be-a13f-4e7d-bb08-8460757f6314 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:39.283472 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-36d9f0be-a13f-4e7d-bb08-8460757f6314 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:39.283970 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-36d9f0be-a13f-4e7d-bb08-8460757f6314 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:39.313461 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-36d9f0be-a13f-4e7d-bb08-8460757f6314 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.046869 Sep 04 01:12:39.314097 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 563/1096] 10.222.0.42 () {60 vars in 1343 bytes} [Sun Sep 4 01:12:39 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b/os-volume_attachments => generated 25 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:12:39.388767 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-c1bbf4fe-4e2c-481b-9977-04f8f5763006 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:39.389866 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-c1bbf4fe-4e2c-481b-9977-04f8f5763006 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:39.398456 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c1bbf4fe-4e2c-481b-9977-04f8f5763006 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:39.398948 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c1bbf4fe-4e2c-481b-9977-04f8f5763006 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:39.399601 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c1bbf4fe-4e2c-481b-9977-04f8f5763006 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:39.400691 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c1bbf4fe-4e2c-481b-9977-04f8f5763006 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:39.401367 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c1bbf4fe-4e2c-481b-9977-04f8f5763006 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:39.401988 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c1bbf4fe-4e2c-481b-9977-04f8f5763006 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:39.422608 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:UPDATE=3,nova_cell1:SELECT=94,nova_cell1:INSERT=3 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:12:39.434205 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-c1bbf4fe-4e2c-481b-9977-04f8f5763006 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Going to try to terminate instance {{(pid=121037) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 04 01:12:39.709123 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-c1bbf4fe-4e2c-481b-9977-04f8f5763006 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "DELETE /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 204 len: 0 microversion: 2.20 time: 0.317867 Sep 04 01:12:39.709123 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 534/1097] 10.222.0.42 () {60 vars in 1280 bytes} [Sun Sep 4 01:12:39 2022] DELETE /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 0 bytes in 319 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 04 01:12:39.714306 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-4c1259fd-9ec0-4320-9c15-bb985db0e09e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:39.714918 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-4c1259fd-9ec0-4320-9c15-bb985db0e09e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:39.721115 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4c1259fd-9ec0-4320-9c15-bb985db0e09e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:39.722030 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4c1259fd-9ec0-4320-9c15-bb985db0e09e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:39.722691 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4c1259fd-9ec0-4320-9c15-bb985db0e09e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:39.724065 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4c1259fd-9ec0-4320-9c15-bb985db0e09e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:39.724687 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4c1259fd-9ec0-4320-9c15-bb985db0e09e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:39.725551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4c1259fd-9ec0-4320-9c15-bb985db0e09e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:39.869733 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-4c1259fd-9ec0-4320-9c15-bb985db0e09e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:39.878395 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-4c1259fd-9ec0-4320-9c15-bb985db0e09e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:39.879840 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-4c1259fd-9ec0-4320-9c15-bb985db0e09e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:39.880986 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-4c1259fd-9ec0-4320-9c15-bb985db0e09e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1722 microversion: 2.20 time: 0.170693 Sep 04 01:12:39.881581 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 564/1098] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:12:39 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1722 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:12:40.908968 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-f0d79f9a-9d0d-469a-a9a8-7f19c8cbd8ec tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:40.909626 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-f0d79f9a-9d0d-469a-a9a8-7f19c8cbd8ec tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:40.916680 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f0d79f9a-9d0d-469a-a9a8-7f19c8cbd8ec tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:40.917202 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f0d79f9a-9d0d-469a-a9a8-7f19c8cbd8ec tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:40.917659 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f0d79f9a-9d0d-469a-a9a8-7f19c8cbd8ec tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:40.918341 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f0d79f9a-9d0d-469a-a9a8-7f19c8cbd8ec tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:40.918792 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f0d79f9a-9d0d-469a-a9a8-7f19c8cbd8ec tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:40.919219 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f0d79f9a-9d0d-469a-a9a8-7f19c8cbd8ec tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:41.055122 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-f0d79f9a-9d0d-469a-a9a8-7f19c8cbd8ec tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:41.063273 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-f0d79f9a-9d0d-469a-a9a8-7f19c8cbd8ec tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:41.065308 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-f0d79f9a-9d0d-469a-a9a8-7f19c8cbd8ec tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:41.066463 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-f0d79f9a-9d0d-469a-a9a8-7f19c8cbd8ec tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1722 microversion: 2.20 time: 0.163581 Sep 04 01:12:41.067015 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 535/1099] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:12:40 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1722 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:12:42.093025 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-0732e74d-19e2-45c8-b1c4-ac339ead17ba tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:42.094362 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-0732e74d-19e2-45c8-b1c4-ac339ead17ba tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:42.106439 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0732e74d-19e2-45c8-b1c4-ac339ead17ba tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:42.106537 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0732e74d-19e2-45c8-b1c4-ac339ead17ba tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:42.109002 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0732e74d-19e2-45c8-b1c4-ac339ead17ba tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:42.109002 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0732e74d-19e2-45c8-b1c4-ac339ead17ba tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:42.109002 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0732e74d-19e2-45c8-b1c4-ac339ead17ba tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:42.109002 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-0732e74d-19e2-45c8-b1c4-ac339ead17ba tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:42.126103 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=100 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:12:42.178646 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-0732e74d-19e2-45c8-b1c4-ac339ead17ba tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:42.189385 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-0732e74d-19e2-45c8-b1c4-ac339ead17ba tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:42.195350 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-0732e74d-19e2-45c8-b1c4-ac339ead17ba tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:42.195350 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-0732e74d-19e2-45c8-b1c4-ac339ead17ba tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1655 microversion: 2.20 time: 0.103970 Sep 04 01:12:42.195350 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 565/1100] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:12:42 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1655 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:12:42.248027 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-063b1b85-5a8a-4afd-bd1b-3348001c13cc req-b0ec72f9-6c32-407d-8652-db8227ec2007 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "da1d8cad-5ea0-48fc-a5af-81eec3d1e94b", "name": "network-vif-deleted", "tag": "19f203f1-14db-4bb0-aaad-c8e57c75d249"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:12:42.258566 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-063b1b85-5a8a-4afd-bd1b-3348001c13cc req-b0ec72f9-6c32-407d-8652-db8227ec2007 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:42.259123 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-063b1b85-5a8a-4afd-bd1b-3348001c13cc req-b0ec72f9-6c32-407d-8652-db8227ec2007 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:42.259616 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-063b1b85-5a8a-4afd-bd1b-3348001c13cc req-b0ec72f9-6c32-407d-8652-db8227ec2007 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:42.271601 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-063b1b85-5a8a-4afd-bd1b-3348001c13cc req-b0ec72f9-6c32-407d-8652-db8227ec2007 service nova] Creating event network-vif-deleted:19f203f1-14db-4bb0-aaad-c8e57c75d249 for instance da1d8cad-5ea0-48fc-a5af-81eec3d1e94b on hv-cinder-maste Sep 04 01:12:42.276610 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-063b1b85-5a8a-4afd-bd1b-3348001c13cc req-b0ec72f9-6c32-407d-8652-db8227ec2007 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032361 Sep 04 01:12:42.277420 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 536/1101] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:12:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:12:43.220047 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-63b131d8-3b82-4701-bcf5-94920d8ad17c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:43.220774 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-63b131d8-3b82-4701-bcf5-94920d8ad17c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:43.228027 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-63b131d8-3b82-4701-bcf5-94920d8ad17c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:43.228523 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-63b131d8-3b82-4701-bcf5-94920d8ad17c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:43.228964 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-63b131d8-3b82-4701-bcf5-94920d8ad17c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:43.229710 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-63b131d8-3b82-4701-bcf5-94920d8ad17c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:43.230225 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-63b131d8-3b82-4701-bcf5-94920d8ad17c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:43.230636 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-63b131d8-3b82-4701-bcf5-94920d8ad17c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:43.267169 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-63b131d8-3b82-4701-bcf5-94920d8ad17c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:43.267651 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-63b131d8-3b82-4701-bcf5-94920d8ad17c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:43.268087 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-63b131d8-3b82-4701-bcf5-94920d8ad17c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:43.268540 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.objects.instance [None req-63b131d8-3b82-4701-bcf5-94920d8ad17c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lazy-loading 'fault' on Instance uuid da1d8cad-5ea0-48fc-a5af-81eec3d1e94b {{(pid=121036) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:12:43.316174 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-63b131d8-3b82-4701-bcf5-94920d8ad17c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:43.325547 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-63b131d8-3b82-4701-bcf5-94920d8ad17c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:43.327063 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-63b131d8-3b82-4701-bcf5-94920d8ad17c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'fbcd4c19e766446188e3b3f41a44e2e4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '55faed42b7f243df9df879d581797f56', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:12:43.328223 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-63b131d8-3b82-4701-bcf5-94920d8ad17c tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 200 len: 1370 microversion: 2.20 time: 0.112420 Sep 04 01:12:43.345379 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 566/1102] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:12:43 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 1370 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 04 01:12:44.356962 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-cb79b55e-3e95-43cd-85e1-20682a2268f8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:44.357968 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-cb79b55e-3e95-43cd-85e1-20682a2268f8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:44.364204 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-cb79b55e-3e95-43cd-85e1-20682a2268f8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:44.364678 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-cb79b55e-3e95-43cd-85e1-20682a2268f8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:44.365132 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-cb79b55e-3e95-43cd-85e1-20682a2268f8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:44.365821 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-cb79b55e-3e95-43cd-85e1-20682a2268f8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:44.366265 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-cb79b55e-3e95-43cd-85e1-20682a2268f8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:44.366989 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-cb79b55e-3e95-43cd-85e1-20682a2268f8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:44.380694 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.wsgi [None req-cb79b55e-3e95-43cd-85e1-20682a2268f8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] HTTP exception thrown: Instance da1d8cad-5ea0-48fc-a5af-81eec3d1e94b could not be found. Sep 04 01:12:44.382484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-cb79b55e-3e95-43cd-85e1-20682a2268f8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Request method failure captured: Sep 04 01:12:44.382484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: request: GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b HTTP/1.1 Sep 04 01:12:44.382484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept: application/json Sep 04 01:12:44.382484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept-Encoding: identity Sep 04 01:12:44.382484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Connection: close Sep 04 01:12:44.382484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Length: 0 Sep 04 01:12:44.382484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Type: application/json Sep 04 01:12:44.382484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Host: 10.222.0.42 Sep 04 01:12:44.382484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Openstack-System-Scope: None Sep 04 01:12:44.382484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: User-Agent: python-urllib3/1.26.12 Sep 04 01:12:44.382484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Auth-Token: gAAAAABjE_nKpA1VnxteF8h4QX2ISAeVrHzayiTS_h5gb61lOBjA6-Oug7W0SDbkxntdMAdvczgEZhwiDWZ1krkg1A5Y324MQzG5qSTO_8yWZlwGNhCpf9OdVOm9oT4bo33eHTJzvu7g_Ei9xW9o9uHJDzE5TNj7NIiad4pBdXs2jBxLyTcMcQ0 Sep 04 01:12:44.382484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Id: None Sep 04 01:12:44.382484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Name: None Sep 04 01:12:44.382484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Identity-Status: Confirmed Sep 04 01:12:44.382484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Is-Admin-Project: True Sep 04 01:12:44.382484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Openstack-Nova-Api-Version: 2.20 Sep 04 01:12:44.382484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Id: default Sep 04 01:12:44.382484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Name: Default Sep 04 01:12:44.382484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Id: 55faed42b7f243df9df879d581797f56 Sep 04 01:12:44.382484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1588860844 Sep 04 01:12:44.382484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Role: reader,member Sep 04 01:12:44.382484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Roles: reader,member Sep 04 01:12:44.384152 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/55faed42b7f243df9df879d581797f56"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_55faed42b7f243df9df879d581797f56", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/55faed42b7f243df9df879d581797f56"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/55faed42b7f243df9df879d581797f56"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:12:44.384152 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1588860844 Sep 04 01:12:44.384152 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Id: 55faed42b7f243df9df879d581797f56 Sep 04 01:12:44.384152 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1588860844 Sep 04 01:12:44.384152 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User: tempest-AttachVolumeShelveTestJSON-1588860844-project Sep 04 01:12:44.384152 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Id: default Sep 04 01:12:44.384152 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Name: Default Sep 04 01:12:44.384152 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Id: fbcd4c19e766446188e3b3f41a44e2e4 Sep 04 01:12:44.384152 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1588860844-project Sep 04 01:12:44.384152 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: method: > Sep 04 01:12:44.384152 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: exception: Instance da1d8cad-5ea0-48fc-a5af-81eec3d1e94b could not be found. Sep 04 01:12:44.384152 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: action_args: {'id': 'da1d8cad-5ea0-48fc-a5af-81eec3d1e94b'} Sep 04 01:12:44.385054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:12:44.385054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:44.385054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:12:44.385054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:12:44.385054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:12:44.385054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:12:44.385054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:12:44.385054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:12:44.385054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:12:44.385054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:12:44.385054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance da1d8cad-5ea0-48fc-a5af-81eec3d1e94b could not be found. Sep 04 01:12:44.385054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:44.385054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:44.385054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:44.385054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:44.385054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:44.385054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:44.385054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:12:44.385054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:12:44.385054 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:12:44.386807 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:12:44.386807 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 04 01:12:44.386807 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 04 01:12:44.386807 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:12:44.386807 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:12:44.386807 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:12:44.386807 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:12:44.386807 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance da1d8cad-5ea0-48fc-a5af-81eec3d1e94b could not be found. Sep 04 01:12:44.386807 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:44.386807 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:44.386807 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:44.386807 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:44.386807 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:44.386807 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:44.386807 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:12:44.386807 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:12:44.386807 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance da1d8cad-5ea0-48fc-a5af-81eec3d1e94b could not be found. Sep 04 01:12:44.386807 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:44.388156 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-cb79b55e-3e95-43cd-85e1-20682a2268f8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Returning 404 to user: Instance da1d8cad-5ea0-48fc-a5af-81eec3d1e94b could not be found. {{(pid=121037) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:12:44.388156 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-cb79b55e-3e95-43cd-85e1-20682a2268f8 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 404 len: 111 microversion: 2.20 time: 0.032021 Sep 04 01:12:44.388156 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 537/1103] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:12:44 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 04 01:12:44.395793 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-e8aae6a5-d2ac-4fe6-a8bf-8259de569847 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '.version_select at 0x7fc10c6a13a0>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:44.473881 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-e8aae6a5-d2ac-4fe6-a8bf-8259de569847 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-419803257" status: 204 len: 0 microversion: 2.20 time: 0.083591 Sep 04 01:12:44.474648 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 567/1104] 10.222.0.42 () {60 vars in 1259 bytes} [Sun Sep 4 01:12:44 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-419803257 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 04 01:12:46.808230 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-705ebce1-b16b-4341-b2e4-27c5b3183d1f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:46.809745 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-705ebce1-b16b-4341-b2e4-27c5b3183d1f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:46.818129 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-705ebce1-b16b-4341-b2e4-27c5b3183d1f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:46.818748 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-705ebce1-b16b-4341-b2e4-27c5b3183d1f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:46.819173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-705ebce1-b16b-4341-b2e4-27c5b3183d1f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:46.820090 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-705ebce1-b16b-4341-b2e4-27c5b3183d1f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:46.820309 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-705ebce1-b16b-4341-b2e4-27c5b3183d1f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:46.820828 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-705ebce1-b16b-4341-b2e4-27c5b3183d1f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:46.837044 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.wsgi [None req-705ebce1-b16b-4341-b2e4-27c5b3183d1f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] HTTP exception thrown: Instance da1d8cad-5ea0-48fc-a5af-81eec3d1e94b could not be found. Sep 04 01:12:46.838647 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-705ebce1-b16b-4341-b2e4-27c5b3183d1f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Request method failure captured: Sep 04 01:12:46.838647 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: request: DELETE /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b HTTP/1.1 Sep 04 01:12:46.838647 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept: application/json Sep 04 01:12:46.838647 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept-Encoding: identity Sep 04 01:12:46.838647 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Connection: close Sep 04 01:12:46.838647 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Length: 0 Sep 04 01:12:46.838647 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Type: application/json Sep 04 01:12:46.838647 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Host: 10.222.0.42 Sep 04 01:12:46.838647 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Openstack-System-Scope: None Sep 04 01:12:46.838647 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: User-Agent: python-urllib3/1.26.12 Sep 04 01:12:46.838647 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Auth-Token: gAAAAABjE_nKpA1VnxteF8h4QX2ISAeVrHzayiTS_h5gb61lOBjA6-Oug7W0SDbkxntdMAdvczgEZhwiDWZ1krkg1A5Y324MQzG5qSTO_8yWZlwGNhCpf9OdVOm9oT4bo33eHTJzvu7g_Ei9xW9o9uHJDzE5TNj7NIiad4pBdXs2jBxLyTcMcQ0 Sep 04 01:12:46.838647 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Id: None Sep 04 01:12:46.838647 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Name: None Sep 04 01:12:46.838647 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Identity-Status: Confirmed Sep 04 01:12:46.838647 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Is-Admin-Project: True Sep 04 01:12:46.838647 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Openstack-Nova-Api-Version: 2.20 Sep 04 01:12:46.838647 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Id: default Sep 04 01:12:46.838647 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Name: Default Sep 04 01:12:46.838647 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Id: 55faed42b7f243df9df879d581797f56 Sep 04 01:12:46.838647 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1588860844 Sep 04 01:12:46.838647 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Role: reader,member Sep 04 01:12:46.838647 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Roles: reader,member Sep 04 01:12:46.840322 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/55faed42b7f243df9df879d581797f56"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_55faed42b7f243df9df879d581797f56", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/55faed42b7f243df9df879d581797f56"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/55faed42b7f243df9df879d581797f56"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:12:46.840322 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1588860844 Sep 04 01:12:46.840322 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Id: 55faed42b7f243df9df879d581797f56 Sep 04 01:12:46.840322 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1588860844 Sep 04 01:12:46.840322 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User: tempest-AttachVolumeShelveTestJSON-1588860844-project Sep 04 01:12:46.840322 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Id: default Sep 04 01:12:46.840322 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Name: Default Sep 04 01:12:46.840322 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Id: fbcd4c19e766446188e3b3f41a44e2e4 Sep 04 01:12:46.840322 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1588860844-project Sep 04 01:12:46.840322 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: method: > Sep 04 01:12:46.840322 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: exception: Instance da1d8cad-5ea0-48fc-a5af-81eec3d1e94b could not be found. Sep 04 01:12:46.840322 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: action_args: {'id': 'da1d8cad-5ea0-48fc-a5af-81eec3d1e94b'} Sep 04 01:12:46.841381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:12:46.841381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:46.841381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:12:46.841381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:12:46.841381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:12:46.841381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:12:46.841381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:12:46.841381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:12:46.841381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:12:46.841381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:12:46.841381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance da1d8cad-5ea0-48fc-a5af-81eec3d1e94b could not be found. Sep 04 01:12:46.841381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:46.841381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:46.841381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:46.841381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:46.841381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:46.841381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:46.841381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:12:46.841381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:12:46.841381 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:12:46.842804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:12:46.842804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 04 01:12:46.842804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 04 01:12:46.842804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 04 01:12:46.842804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 04 01:12:46.842804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:12:46.842804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:12:46.842804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:12:46.842804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:12:46.842804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance da1d8cad-5ea0-48fc-a5af-81eec3d1e94b could not be found. Sep 04 01:12:46.842804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:46.842804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:46.842804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:46.842804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:46.842804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:46.842804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:46.842804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:12:46.842804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:12:46.842804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance da1d8cad-5ea0-48fc-a5af-81eec3d1e94b could not be found. Sep 04 01:12:46.842804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:46.844249 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-705ebce1-b16b-4341-b2e4-27c5b3183d1f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Returning 404 to user: Instance da1d8cad-5ea0-48fc-a5af-81eec3d1e94b could not be found. {{(pid=121037) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:12:46.844249 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-705ebce1-b16b-4341-b2e4-27c5b3183d1f tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "DELETE /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 404 len: 111 microversion: 2.20 time: 0.037612 Sep 04 01:12:46.844249 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 538/1105] 10.222.0.42 () {60 vars in 1280 bytes} [Sun Sep 4 01:12:46 2022] DELETE /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 04 01:12:46.849381 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-ecb29a89-0754-4b89-9d77-9b19514c9430 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:46.850196 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-ecb29a89-0754-4b89-9d77-9b19514c9430 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: da1d8cad-5ea0-48fc-a5af-81eec3d1e94b] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:46.857481 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ecb29a89-0754-4b89-9d77-9b19514c9430 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:46.857968 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ecb29a89-0754-4b89-9d77-9b19514c9430 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:46.858406 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ecb29a89-0754-4b89-9d77-9b19514c9430 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:46.859141 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ecb29a89-0754-4b89-9d77-9b19514c9430 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:46.859630 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ecb29a89-0754-4b89-9d77-9b19514c9430 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:46.860026 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ecb29a89-0754-4b89-9d77-9b19514c9430 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:46.874291 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.wsgi [None req-ecb29a89-0754-4b89-9d77-9b19514c9430 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] HTTP exception thrown: Instance da1d8cad-5ea0-48fc-a5af-81eec3d1e94b could not be found. Sep 04 01:12:46.878216 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-ecb29a89-0754-4b89-9d77-9b19514c9430 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Request method failure captured: Sep 04 01:12:46.878216 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: request: GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b HTTP/1.1 Sep 04 01:12:46.878216 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept: application/json Sep 04 01:12:46.878216 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept-Encoding: identity Sep 04 01:12:46.878216 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Connection: close Sep 04 01:12:46.878216 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Length: 0 Sep 04 01:12:46.878216 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Type: application/json Sep 04 01:12:46.878216 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Host: 10.222.0.42 Sep 04 01:12:46.878216 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Openstack-System-Scope: None Sep 04 01:12:46.878216 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: User-Agent: python-urllib3/1.26.12 Sep 04 01:12:46.878216 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Auth-Token: gAAAAABjE_nKpA1VnxteF8h4QX2ISAeVrHzayiTS_h5gb61lOBjA6-Oug7W0SDbkxntdMAdvczgEZhwiDWZ1krkg1A5Y324MQzG5qSTO_8yWZlwGNhCpf9OdVOm9oT4bo33eHTJzvu7g_Ei9xW9o9uHJDzE5TNj7NIiad4pBdXs2jBxLyTcMcQ0 Sep 04 01:12:46.878216 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Id: None Sep 04 01:12:46.878216 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Name: None Sep 04 01:12:46.878216 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Identity-Status: Confirmed Sep 04 01:12:46.878216 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Is-Admin-Project: True Sep 04 01:12:46.878216 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Openstack-Nova-Api-Version: 2.20 Sep 04 01:12:46.878216 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Id: default Sep 04 01:12:46.878216 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Name: Default Sep 04 01:12:46.878216 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Id: 55faed42b7f243df9df879d581797f56 Sep 04 01:12:46.878216 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1588860844 Sep 04 01:12:46.878216 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Role: reader,member Sep 04 01:12:46.878216 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Roles: reader,member Sep 04 01:12:46.879726 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/55faed42b7f243df9df879d581797f56"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_55faed42b7f243df9df879d581797f56", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/55faed42b7f243df9df879d581797f56"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/55faed42b7f243df9df879d581797f56"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:12:46.879726 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1588860844 Sep 04 01:12:46.879726 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Id: 55faed42b7f243df9df879d581797f56 Sep 04 01:12:46.879726 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1588860844 Sep 04 01:12:46.879726 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User: tempest-AttachVolumeShelveTestJSON-1588860844-project Sep 04 01:12:46.879726 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Id: default Sep 04 01:12:46.879726 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Name: Default Sep 04 01:12:46.879726 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Id: fbcd4c19e766446188e3b3f41a44e2e4 Sep 04 01:12:46.879726 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1588860844-project Sep 04 01:12:46.879726 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: method: > Sep 04 01:12:46.879726 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: exception: Instance da1d8cad-5ea0-48fc-a5af-81eec3d1e94b could not be found. Sep 04 01:12:46.879726 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: action_args: {'id': 'da1d8cad-5ea0-48fc-a5af-81eec3d1e94b'} Sep 04 01:12:46.880563 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:12:46.880563 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:46.880563 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:12:46.880563 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:12:46.880563 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:12:46.880563 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:12:46.880563 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:12:46.880563 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:12:46.880563 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:12:46.880563 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:12:46.880563 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance da1d8cad-5ea0-48fc-a5af-81eec3d1e94b could not be found. Sep 04 01:12:46.880563 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:46.880563 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:46.880563 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:46.880563 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:46.880563 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:46.880563 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:46.880563 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:12:46.880563 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:12:46.880563 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:12:46.881829 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:12:46.881829 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 04 01:12:46.881829 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 04 01:12:46.881829 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:12:46.881829 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:12:46.881829 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:12:46.881829 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:12:46.881829 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance da1d8cad-5ea0-48fc-a5af-81eec3d1e94b could not be found. Sep 04 01:12:46.881829 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:46.881829 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:46.881829 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:46.881829 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:46.881829 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:46.881829 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:46.881829 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:12:46.881829 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:12:46.881829 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance da1d8cad-5ea0-48fc-a5af-81eec3d1e94b could not be found. Sep 04 01:12:46.881829 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:46.883110 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-ecb29a89-0754-4b89-9d77-9b19514c9430 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Returning 404 to user: Instance da1d8cad-5ea0-48fc-a5af-81eec3d1e94b could not be found. {{(pid=121036) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:12:46.883110 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-ecb29a89-0754-4b89-9d77-9b19514c9430 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b" status: 404 len: 111 microversion: 2.20 time: 0.032363 Sep 04 01:12:46.883110 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 568/1106] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:12:46 2022] GET /compute/v2.1/servers/da1d8cad-5ea0-48fc-a5af-81eec3d1e94b => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 04 01:12:51.287917 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-5a5d2229-6aa5-4a3a-b131-52828146782e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:51.288433 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-5a5d2229-6aa5-4a3a-b131-52828146782e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:51.294999 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5a5d2229-6aa5-4a3a-b131-52828146782e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:51.295471 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5a5d2229-6aa5-4a3a-b131-52828146782e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:51.295897 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5a5d2229-6aa5-4a3a-b131-52828146782e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:51.296575 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5a5d2229-6aa5-4a3a-b131-52828146782e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:51.297022 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5a5d2229-6aa5-4a3a-b131-52828146782e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:51.297464 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5a5d2229-6aa5-4a3a-b131-52828146782e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:51.311522 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.wsgi [None req-5a5d2229-6aa5-4a3a-b131-52828146782e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] HTTP exception thrown: Instance 49be5a0b-f748-486e-a546-c07a8e8451a3 could not be found. Sep 04 01:12:51.312897 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-5a5d2229-6aa5-4a3a-b131-52828146782e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Request method failure captured: Sep 04 01:12:51.312897 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: request: DELETE /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 HTTP/1.1 Sep 04 01:12:51.312897 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept: application/json Sep 04 01:12:51.312897 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept-Encoding: identity Sep 04 01:12:51.312897 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Connection: close Sep 04 01:12:51.312897 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Length: 0 Sep 04 01:12:51.312897 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Type: application/json Sep 04 01:12:51.312897 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Host: 10.222.0.42 Sep 04 01:12:51.312897 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Openstack-System-Scope: None Sep 04 01:12:51.312897 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: User-Agent: python-urllib3/1.26.12 Sep 04 01:12:51.312897 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Auth-Token: gAAAAABjE_nKpA1VnxteF8h4QX2ISAeVrHzayiTS_h5gb61lOBjA6-Oug7W0SDbkxntdMAdvczgEZhwiDWZ1krkg1A5Y324MQzG5qSTO_8yWZlwGNhCpf9OdVOm9oT4bo33eHTJzvu7g_Ei9xW9o9uHJDzE5TNj7NIiad4pBdXs2jBxLyTcMcQ0 Sep 04 01:12:51.312897 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Id: None Sep 04 01:12:51.312897 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Name: None Sep 04 01:12:51.312897 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Identity-Status: Confirmed Sep 04 01:12:51.312897 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Is-Admin-Project: True Sep 04 01:12:51.312897 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Openstack-Nova-Api-Version: 2.20 Sep 04 01:12:51.312897 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Id: default Sep 04 01:12:51.312897 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Name: Default Sep 04 01:12:51.312897 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Id: 55faed42b7f243df9df879d581797f56 Sep 04 01:12:51.312897 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1588860844 Sep 04 01:12:51.312897 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Role: reader,member Sep 04 01:12:51.312897 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Roles: reader,member Sep 04 01:12:51.314572 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/55faed42b7f243df9df879d581797f56"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_55faed42b7f243df9df879d581797f56", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/55faed42b7f243df9df879d581797f56"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/55faed42b7f243df9df879d581797f56"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:12:51.314572 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1588860844 Sep 04 01:12:51.314572 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Id: 55faed42b7f243df9df879d581797f56 Sep 04 01:12:51.314572 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1588860844 Sep 04 01:12:51.314572 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User: tempest-AttachVolumeShelveTestJSON-1588860844-project Sep 04 01:12:51.314572 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Id: default Sep 04 01:12:51.314572 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Name: Default Sep 04 01:12:51.314572 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Id: fbcd4c19e766446188e3b3f41a44e2e4 Sep 04 01:12:51.314572 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1588860844-project Sep 04 01:12:51.314572 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: method: > Sep 04 01:12:51.314572 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: exception: Instance 49be5a0b-f748-486e-a546-c07a8e8451a3 could not be found. Sep 04 01:12:51.314572 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: action_args: {'id': '49be5a0b-f748-486e-a546-c07a8e8451a3'} Sep 04 01:12:51.315435 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:12:51.315435 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:51.315435 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:12:51.315435 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:12:51.315435 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:12:51.315435 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:12:51.315435 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:12:51.315435 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:12:51.315435 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:12:51.315435 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:12:51.315435 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 49be5a0b-f748-486e-a546-c07a8e8451a3 could not be found. Sep 04 01:12:51.315435 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:51.315435 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:51.315435 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:51.315435 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:51.315435 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:51.315435 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:51.315435 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:12:51.315435 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:12:51.315435 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:12:51.317654 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:12:51.317654 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 04 01:12:51.317654 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 04 01:12:51.317654 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 04 01:12:51.317654 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 04 01:12:51.317654 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:12:51.317654 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:12:51.317654 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:12:51.317654 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:12:51.317654 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 49be5a0b-f748-486e-a546-c07a8e8451a3 could not be found. Sep 04 01:12:51.317654 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:51.317654 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:51.317654 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:51.317654 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:51.317654 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:51.317654 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:51.317654 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:12:51.317654 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:12:51.317654 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 49be5a0b-f748-486e-a546-c07a8e8451a3 could not be found. Sep 04 01:12:51.317654 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:12:51.321524 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-5a5d2229-6aa5-4a3a-b131-52828146782e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Returning 404 to user: Instance 49be5a0b-f748-486e-a546-c07a8e8451a3 could not be found. {{(pid=121037) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:12:51.321524 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-5a5d2229-6aa5-4a3a-b131-52828146782e tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "DELETE /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 404 len: 111 microversion: 2.20 time: 0.030323 Sep 04 01:12:51.321524 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 539/1107] 10.222.0.42 () {60 vars in 1280 bytes} [Sun Sep 4 01:12:51 2022] DELETE /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 04 01:12:51.324364 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-888d6018-80b9-4124-b39c-f006f1009782 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:51.325044 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-888d6018-80b9-4124-b39c-f006f1009782 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] [instance: 49be5a0b-f748-486e-a546-c07a8e8451a3] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:12:51.327671 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_api:SELECT=51,nova_api:DELETE=1 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:12:51.332790 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-888d6018-80b9-4124-b39c-f006f1009782 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:51.333393 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-888d6018-80b9-4124-b39c-f006f1009782 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:51.333823 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-888d6018-80b9-4124-b39c-f006f1009782 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:51.340303 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-888d6018-80b9-4124-b39c-f006f1009782 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:12:51.340303 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-888d6018-80b9-4124-b39c-f006f1009782 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:12:51.340303 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-888d6018-80b9-4124-b39c-f006f1009782 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:12:51.349973 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.wsgi [None req-888d6018-80b9-4124-b39c-f006f1009782 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] HTTP exception thrown: Instance 49be5a0b-f748-486e-a546-c07a8e8451a3 could not be found. Sep 04 01:12:51.351551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-888d6018-80b9-4124-b39c-f006f1009782 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Request method failure captured: Sep 04 01:12:51.351551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: request: GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 HTTP/1.1 Sep 04 01:12:51.351551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept: application/json Sep 04 01:12:51.351551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept-Encoding: identity Sep 04 01:12:51.351551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Connection: close Sep 04 01:12:51.351551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Length: 0 Sep 04 01:12:51.351551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Type: application/json Sep 04 01:12:51.351551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Host: 10.222.0.42 Sep 04 01:12:51.351551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Openstack-System-Scope: None Sep 04 01:12:51.351551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: User-Agent: python-urllib3/1.26.12 Sep 04 01:12:51.351551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Auth-Token: gAAAAABjE_nKpA1VnxteF8h4QX2ISAeVrHzayiTS_h5gb61lOBjA6-Oug7W0SDbkxntdMAdvczgEZhwiDWZ1krkg1A5Y324MQzG5qSTO_8yWZlwGNhCpf9OdVOm9oT4bo33eHTJzvu7g_Ei9xW9o9uHJDzE5TNj7NIiad4pBdXs2jBxLyTcMcQ0 Sep 04 01:12:51.351551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Id: None Sep 04 01:12:51.351551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Name: None Sep 04 01:12:51.351551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Identity-Status: Confirmed Sep 04 01:12:51.351551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Is-Admin-Project: True Sep 04 01:12:51.351551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Openstack-Nova-Api-Version: 2.20 Sep 04 01:12:51.351551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Id: default Sep 04 01:12:51.351551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Name: Default Sep 04 01:12:51.351551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Id: 55faed42b7f243df9df879d581797f56 Sep 04 01:12:51.351551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1588860844 Sep 04 01:12:51.351551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Role: reader,member Sep 04 01:12:51.351551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Roles: reader,member Sep 04 01:12:51.353141 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/55faed42b7f243df9df879d581797f56"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_55faed42b7f243df9df879d581797f56", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/55faed42b7f243df9df879d581797f56"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/55faed42b7f243df9df879d581797f56"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:12:51.353141 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1588860844 Sep 04 01:12:51.353141 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Id: 55faed42b7f243df9df879d581797f56 Sep 04 01:12:51.353141 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1588860844 Sep 04 01:12:51.353141 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User: tempest-AttachVolumeShelveTestJSON-1588860844-project Sep 04 01:12:51.353141 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Id: default Sep 04 01:12:51.353141 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Name: Default Sep 04 01:12:51.353141 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Id: fbcd4c19e766446188e3b3f41a44e2e4 Sep 04 01:12:51.353141 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1588860844-project Sep 04 01:12:51.353141 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: method: > Sep 04 01:12:51.353141 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: exception: Instance 49be5a0b-f748-486e-a546-c07a8e8451a3 could not be found. Sep 04 01:12:51.353141 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: action_args: {'id': '49be5a0b-f748-486e-a546-c07a8e8451a3'} Sep 04 01:12:51.354180 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:12:51.354180 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:51.354180 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:12:51.354180 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:12:51.354180 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:12:51.354180 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:12:51.354180 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:12:51.354180 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:12:51.354180 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:12:51.354180 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:12:51.354180 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 49be5a0b-f748-486e-a546-c07a8e8451a3 could not be found. Sep 04 01:12:51.354180 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:51.354180 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:51.354180 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:51.354180 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:51.354180 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:51.354180 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:51.354180 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:12:51.354180 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:12:51.354180 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:12:51.355826 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:12:51.355826 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 04 01:12:51.355826 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 04 01:12:51.355826 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:12:51.355826 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:12:51.355826 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:12:51.355826 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:12:51.355826 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 49be5a0b-f748-486e-a546-c07a8e8451a3 could not be found. Sep 04 01:12:51.355826 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:51.355826 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:12:51.355826 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:51.355826 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:12:51.355826 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:12:51.355826 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:12:51.355826 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:12:51.355826 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:12:51.355826 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 49be5a0b-f748-486e-a546-c07a8e8451a3 could not be found. Sep 04 01:12:51.355826 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:12:51.356911 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-888d6018-80b9-4124-b39c-f006f1009782 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] Returning 404 to user: Instance 49be5a0b-f748-486e-a546-c07a8e8451a3 could not be found. {{(pid=121036) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:12:51.356911 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-888d6018-80b9-4124-b39c-f006f1009782 tempest-AttachVolumeShelveTestJSON-1588860844 tempest-AttachVolumeShelveTestJSON-1588860844-project] 10.222.0.42 "GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3" status: 404 len: 111 microversion: 2.20 time: 0.033186 Sep 04 01:12:51.356911 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 569/1108] 10.222.0.42 () {60 vars in 1277 bytes} [Sun Sep 4 01:12:51 2022] GET /compute/v2.1/servers/49be5a0b-f748-486e-a546-c07a8e8451a3 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 04 01:12:56.980332 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-48c103b7-e26b-4504-b34d-3563df6e0910 tempest-VolumesGetTestJSON-1758823095 tempest-VolumesGetTestJSON-1758823095-project] Action: 'create', calling method: .version_select at 0x7fc10c614ee0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-321613715", "metadata": {"Type": "work"}}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:12:57.701177 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-48c103b7-e26b-4504-b34d-3563df6e0910 tempest-VolumesGetTestJSON-1758823095 tempest-VolumesGetTestJSON-1758823095-project] 10.222.0.42 "POST /compute/v2.1/os-volumes" status: 200 len: 352 microversion: 2.1 time: 0.832785 Sep 04 01:12:57.701835 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 540/1109] 10.222.0.42 () {60 vars in 1156 bytes} [Sun Sep 4 01:12:56 2022] POST /compute/v2.1/os-volumes => generated 352 bytes in 834 msecs (HTTP/1.1 200) 10 headers in 448 bytes (2 switches on core 0) Sep 04 01:12:57.715652 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-8ae432f9-9872-464a-b8a4-fcbdcbd3d3cc tempest-VolumesGetTestJSON-1758823095 tempest-VolumesGetTestJSON-1758823095-project] Calling method '.version_select at 0x7fc10c724790>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:57.770252 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-8ae432f9-9872-464a-b8a4-fcbdcbd3d3cc tempest-VolumesGetTestJSON-1758823095 tempest-VolumesGetTestJSON-1758823095-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/644d810a-5069-4082-a7e5-b63fe3e4acf5" status: 200 len: 352 microversion: 2.1 time: 0.058615 Sep 04 01:12:57.771964 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 570/1110] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Sep 4 01:12:57 2022] GET /compute/v2.1/os-volumes/644d810a-5069-4082-a7e5-b63fe3e4acf5 => generated 352 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:12:58.792154 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-9c0bd437-ce78-4139-a3f4-dba61090eb4d tempest-VolumesGetTestJSON-1758823095 tempest-VolumesGetTestJSON-1758823095-project] Calling method '.version_select at 0x7fc10c6da430>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:58.844241 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-9c0bd437-ce78-4139-a3f4-dba61090eb4d tempest-VolumesGetTestJSON-1758823095 tempest-VolumesGetTestJSON-1758823095-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/644d810a-5069-4082-a7e5-b63fe3e4acf5" status: 200 len: 353 microversion: 2.1 time: 0.055840 Sep 04 01:12:58.844905 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 541/1111] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Sep 4 01:12:58 2022] GET /compute/v2.1/os-volumes/644d810a-5069-4082-a7e5-b63fe3e4acf5 => generated 353 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:12:58.860064 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-5c168174-b0b4-4f3e-839a-8165b102afe9 tempest-VolumesGetTestJSON-1758823095 tempest-VolumesGetTestJSON-1758823095-project] Calling method '.version_select at 0x7fc10c72e1f0>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:58.915141 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-5c168174-b0b4-4f3e-839a-8165b102afe9 tempest-VolumesGetTestJSON-1758823095 tempest-VolumesGetTestJSON-1758823095-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/644d810a-5069-4082-a7e5-b63fe3e4acf5" status: 200 len: 353 microversion: 2.1 time: 0.059610 Sep 04 01:12:58.915744 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 571/1112] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Sep 4 01:12:58 2022] GET /compute/v2.1/os-volumes/644d810a-5069-4082-a7e5-b63fe3e4acf5 => generated 353 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:12:58.933940 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-dcf58dff-4b8c-48fc-99d7-41464c700355 tempest-VolumesGetTestJSON-1758823095 tempest-VolumesGetTestJSON-1758823095-project] Calling method '.version_select at 0x7fc10c84cf70>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:59.137581 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-dcf58dff-4b8c-48fc-99d7-41464c700355 tempest-VolumesGetTestJSON-1758823095 tempest-VolumesGetTestJSON-1758823095-project] 10.222.0.42 "DELETE /compute/v2.1/os-volumes/644d810a-5069-4082-a7e5-b63fe3e4acf5" status: 202 len: 0 microversion: 2.1 time: 0.205777 Sep 04 01:12:59.137581 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 542/1113] 10.222.0.42 () {58 vars in 1248 bytes} [Sun Sep 4 01:12:58 2022] DELETE /compute/v2.1/os-volumes/644d810a-5069-4082-a7e5-b63fe3e4acf5 => generated 0 bytes in 207 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 04 01:12:59.145401 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-f36a2b65-813e-4294-a07d-3a23862e8a1e tempest-VolumesGetTestJSON-1758823095 tempest-VolumesGetTestJSON-1758823095-project] Calling method '.version_select at 0x7fc10c690310>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:12:59.207940 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-f36a2b65-813e-4294-a07d-3a23862e8a1e tempest-VolumesGetTestJSON-1758823095 tempest-VolumesGetTestJSON-1758823095-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/644d810a-5069-4082-a7e5-b63fe3e4acf5" status: 200 len: 352 microversion: 2.1 time: 0.055164 Sep 04 01:12:59.207940 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 572/1114] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Sep 4 01:12:59 2022] GET /compute/v2.1/os-volumes/644d810a-5069-4082-a7e5-b63fe3e4acf5 => generated 352 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:13:00.220258 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-dedd7771-8b00-44ed-a60f-b4f5397f9ba9 tempest-VolumesGetTestJSON-1758823095 tempest-VolumesGetTestJSON-1758823095-project] Calling method '.version_select at 0x7fc10c6395e0>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:13:00.249171 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.wsgi [None req-dedd7771-8b00-44ed-a60f-b4f5397f9ba9 tempest-VolumesGetTestJSON-1758823095 tempest-VolumesGetTestJSON-1758823095-project] HTTP exception thrown: Volume 644d810a-5069-4082-a7e5-b63fe3e4acf5 could not be found. Sep 04 01:13:00.251173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-dedd7771-8b00-44ed-a60f-b4f5397f9ba9 tempest-VolumesGetTestJSON-1758823095 tempest-VolumesGetTestJSON-1758823095-project] Request method failure captured: Sep 04 01:13:00.251173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: request: GET /compute/v2.1/os-volumes/644d810a-5069-4082-a7e5-b63fe3e4acf5 HTTP/1.1 Sep 04 01:13:00.251173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept: application/json Sep 04 01:13:00.251173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept-Encoding: identity Sep 04 01:13:00.251173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Connection: close Sep 04 01:13:00.251173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Length: 0 Sep 04 01:13:00.251173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Type: application/json Sep 04 01:13:00.251173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Host: 10.222.0.42 Sep 04 01:13:00.251173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Openstack-System-Scope: None Sep 04 01:13:00.251173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: User-Agent: python-urllib3/1.26.12 Sep 04 01:13:00.251173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Auth-Token: gAAAAABjE_uYWopdVUo4_6Tw6QvMq2jTWOOHZGMPg5wm3dp31CXAX86POjiCxeBiR00cYTDw-j4wcY0IBfbeV1aWbbWN7D6SNxhlx65MWEGcl4MHa2MdP4EhskXD_pmroOMhZNEmy-_LAQHdukNhgHdINSvQo9ZALfLcu9lPcRW_aAdiTHMqYKw Sep 04 01:13:00.251173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Id: None Sep 04 01:13:00.251173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Name: None Sep 04 01:13:00.251173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Identity-Status: Confirmed Sep 04 01:13:00.251173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Is-Admin-Project: True Sep 04 01:13:00.251173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Id: default Sep 04 01:13:00.251173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Name: Default Sep 04 01:13:00.251173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Id: 83428cdb97544c7b8ea4bf9a69d7106f Sep 04 01:13:00.251173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Name: tempest-VolumesGetTestJSON-1758823095 Sep 04 01:13:00.251173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Role: reader,member Sep 04 01:13:00.251173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Roles: reader,member Sep 04 01:13:00.252894 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/83428cdb97544c7b8ea4bf9a69d7106f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_83428cdb97544c7b8ea4bf9a69d7106f", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/83428cdb97544c7b8ea4bf9a69d7106f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/83428cdb97544c7b8ea4bf9a69d7106f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:13:00.252894 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant: tempest-VolumesGetTestJSON-1758823095 Sep 04 01:13:00.252894 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Id: 83428cdb97544c7b8ea4bf9a69d7106f Sep 04 01:13:00.252894 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Name: tempest-VolumesGetTestJSON-1758823095 Sep 04 01:13:00.252894 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User: tempest-VolumesGetTestJSON-1758823095-project Sep 04 01:13:00.252894 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Id: default Sep 04 01:13:00.252894 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Name: Default Sep 04 01:13:00.252894 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Id: efa78ca640474aafbc99c03af1aca932 Sep 04 01:13:00.252894 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Name: tempest-VolumesGetTestJSON-1758823095-project Sep 04 01:13:00.252894 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: method: Sep 04 01:13:00.252894 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: exception: Volume 644d810a-5069-4082-a7e5-b63fe3e4acf5 could not be found. Sep 04 01:13:00.252894 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: action_args: {'id': '644d810a-5069-4082-a7e5-b63fe3e4acf5'} Sep 04 01:13:00.252894 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:13:00.254219 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:13:00.254219 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 04 01:13:00.254219 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 04 01:13:00.254219 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 04 01:13:00.254219 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 04 01:13:00.254219 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 04 01:13:00.254219 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 04 01:13:00.254219 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 04 01:13:00.254219 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 04 01:13:00.254219 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 04 01:13:00.254219 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 04 01:13:00.254219 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 04 01:13:00.254219 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 04 01:13:00.254219 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 04 01:13:00.254219 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 04 01:13:00.254219 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 644d810a-5069-4082-a7e5-b63fe3e4acf5 could not be found. (HTTP 404) (Request-ID: req-54d4822f-feb0-45e2-91fb-71eeff4c3280) Sep 04 01:13:00.254219 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:13:00.254219 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:13:00.254219 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:13:00.255754 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:13:00.255754 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 04 01:13:00.255754 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 04 01:13:00.255754 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 04 01:13:00.255754 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 04 01:13:00.255754 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 04 01:13:00.255754 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 04 01:13:00.255754 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 04 01:13:00.255754 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 04 01:13:00.255754 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 04 01:13:00.255754 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 04 01:13:00.255754 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 04 01:13:00.255754 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 04 01:13:00.255754 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 04 01:13:00.255754 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 04 01:13:00.255754 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 04 01:13:00.255754 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 04 01:13:00.255754 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 04 01:13:00.255754 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 04 01:13:00.258409 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 04 01:13:00.258409 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 04 01:13:00.258409 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 04 01:13:00.258409 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 04 01:13:00.258409 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 644d810a-5069-4082-a7e5-b63fe3e4acf5 could not be found. Sep 04 01:13:00.258409 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:13:00.258409 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:13:00.258409 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:13:00.258409 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:13:00.258409 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:13:00.258409 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:13:00.258409 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:13:00.258409 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:13:00.258409 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 04 01:13:00.258409 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 04 01:13:00.258409 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:13:00.258409 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:13:00.258409 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 04 01:13:00.258409 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:13:00.259996 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 644d810a-5069-4082-a7e5-b63fe3e4acf5 could not be found. Sep 04 01:13:00.259996 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:13:00.259996 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:13:00.259996 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:13:00.259996 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:13:00.259996 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:13:00.259996 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:13:00.259996 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:13:00.259996 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:13:00.259996 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 644d810a-5069-4082-a7e5-b63fe3e4acf5 could not be found. Sep 04 01:13:00.259996 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:13:00.259996 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-dedd7771-8b00-44ed-a60f-b4f5397f9ba9 tempest-VolumesGetTestJSON-1758823095 tempest-VolumesGetTestJSON-1758823095-project] Returning 404 to user: Volume 644d810a-5069-4082-a7e5-b63fe3e4acf5 could not be found. {{(pid=121037) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:13:00.259996 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-dedd7771-8b00-44ed-a60f-b4f5397f9ba9 tempest-VolumesGetTestJSON-1758823095 tempest-VolumesGetTestJSON-1758823095-project] 10.222.0.42 "GET /compute/v2.1/os-volumes/644d810a-5069-4082-a7e5-b63fe3e4acf5" status: 404 len: 109 microversion: 2.1 time: 0.037030 Sep 04 01:13:00.259996 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 543/1115] 10.222.0.42 () {58 vars in 1245 bytes} [Sun Sep 4 01:13:00 2022] GET /compute/v2.1/os-volumes/644d810a-5069-4082-a7e5-b63fe3e4acf5 => generated 109 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 04 01:13:51.872124 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=16 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:13:51.873522 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_api:SELECT=1 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:13:51.875759 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_api:SELECT=50,nova_api:UPDATE=2,nova_api:DELETE=1 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:13:51.876659 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=25,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:13:51.879367 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-e29482fb-f47e-4fd8-98f4-28832327f7ba tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:13:51.879367 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-e29482fb-f47e-4fd8-98f4-28832327f7ba tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:13:51.892273 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e29482fb-f47e-4fd8-98f4-28832327f7ba tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:13:51.892907 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e29482fb-f47e-4fd8-98f4-28832327f7ba tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:13:51.893414 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e29482fb-f47e-4fd8-98f4-28832327f7ba tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:13:51.894105 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e29482fb-f47e-4fd8-98f4-28832327f7ba tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:13:51.894562 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e29482fb-f47e-4fd8-98f4-28832327f7ba tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:13:51.894988 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e29482fb-f47e-4fd8-98f4-28832327f7ba tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:13:51.968348 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-e29482fb-f47e-4fd8-98f4-28832327f7ba tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Going to try to terminate instance {{(pid=121036) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 04 01:13:52.069850 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-fb808cf8-9b28-4f05-971d-a646329f235d req-b885d98b-2390-4f7c-a6e0-7d150d84de75 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "74df11dc-31bd-4745-83bc-4b3c430d02ff", "tag": "d072bfcf-41bd-430e-800c-1d1a3b17d915"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:13:52.080827 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-fb808cf8-9b28-4f05-971d-a646329f235d req-b885d98b-2390-4f7c-a6e0-7d150d84de75 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:13:52.081452 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-fb808cf8-9b28-4f05-971d-a646329f235d req-b885d98b-2390-4f7c-a6e0-7d150d84de75 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:13:52.081895 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-fb808cf8-9b28-4f05-971d-a646329f235d req-b885d98b-2390-4f7c-a6e0-7d150d84de75 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:13:52.094123 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-fb808cf8-9b28-4f05-971d-a646329f235d req-b885d98b-2390-4f7c-a6e0-7d150d84de75 service nova] Creating event network-changed:d072bfcf-41bd-430e-800c-1d1a3b17d915 for instance 74df11dc-31bd-4745-83bc-4b3c430d02ff on hv-cinder-maste Sep 04 01:13:52.099644 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-fb808cf8-9b28-4f05-971d-a646329f235d req-b885d98b-2390-4f7c-a6e0-7d150d84de75 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.225883 Sep 04 01:13:52.100326 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 544/1116] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:13:51 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:13:52.329959 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-e29482fb-f47e-4fd8-98f4-28832327f7ba tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "DELETE /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 204 len: 0 microversion: 2.1 time: 0.459391 Sep 04 01:13:52.331377 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 573/1117] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 4 01:13:51 2022] DELETE /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 0 bytes in 462 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 04 01:13:52.341518 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-9a90f0bd-2ac6-487c-b7dc-333f0a665297 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:13:52.342441 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-9a90f0bd-2ac6-487c-b7dc-333f0a665297 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:13:52.350057 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9a90f0bd-2ac6-487c-b7dc-333f0a665297 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:13:52.350604 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9a90f0bd-2ac6-487c-b7dc-333f0a665297 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:13:52.350981 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9a90f0bd-2ac6-487c-b7dc-333f0a665297 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:13:52.351674 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9a90f0bd-2ac6-487c-b7dc-333f0a665297 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:13:52.352128 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9a90f0bd-2ac6-487c-b7dc-333f0a665297 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:13:52.352551 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9a90f0bd-2ac6-487c-b7dc-333f0a665297 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:13:52.492298 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-9a90f0bd-2ac6-487c-b7dc-333f0a665297 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:13:52.500872 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-9a90f0bd-2ac6-487c-b7dc-333f0a665297 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1568 microversion: 2.1 time: 0.164934 Sep 04 01:13:52.501473 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 545/1118] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:13:52 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1568 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:13:53.525355 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-16389cb2-7c58-4d17-af26-56036b40898b tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:13:53.526051 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-16389cb2-7c58-4d17-af26-56036b40898b tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:13:53.534593 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-16389cb2-7c58-4d17-af26-56036b40898b tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:13:53.535101 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-16389cb2-7c58-4d17-af26-56036b40898b tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:13:53.535730 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-16389cb2-7c58-4d17-af26-56036b40898b tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:13:53.536610 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-16389cb2-7c58-4d17-af26-56036b40898b tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:13:53.537066 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-16389cb2-7c58-4d17-af26-56036b40898b tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:13:53.537512 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-16389cb2-7c58-4d17-af26-56036b40898b tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:13:53.677852 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-16389cb2-7c58-4d17-af26-56036b40898b tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:13:53.686350 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-16389cb2-7c58-4d17-af26-56036b40898b tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1568 microversion: 2.1 time: 0.165489 Sep 04 01:13:53.686919 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 574/1119] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:13:53 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1568 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:13:54.711027 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-17a3cf9a-0554-4418-8926-1dac122f9870 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:13:54.711893 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-17a3cf9a-0554-4418-8926-1dac122f9870 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:13:54.719804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-17a3cf9a-0554-4418-8926-1dac122f9870 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:13:54.720289 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-17a3cf9a-0554-4418-8926-1dac122f9870 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:13:54.720726 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-17a3cf9a-0554-4418-8926-1dac122f9870 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:13:54.721785 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-17a3cf9a-0554-4418-8926-1dac122f9870 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:13:54.722240 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-17a3cf9a-0554-4418-8926-1dac122f9870 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:13:54.722667 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-17a3cf9a-0554-4418-8926-1dac122f9870 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:13:54.904404 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-17a3cf9a-0554-4418-8926-1dac122f9870 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:13:54.912161 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-17a3cf9a-0554-4418-8926-1dac122f9870 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1568 microversion: 2.1 time: 0.205859 Sep 04 01:13:54.912721 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 546/1120] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:13:54 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1568 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:13:55.936273 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-cccfea7d-8fa3-48d7-b7d7-5d8fc7445040 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:13:55.936885 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-cccfea7d-8fa3-48d7-b7d7-5d8fc7445040 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:13:55.943282 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cccfea7d-8fa3-48d7-b7d7-5d8fc7445040 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:13:55.943754 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cccfea7d-8fa3-48d7-b7d7-5d8fc7445040 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:13:55.944184 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cccfea7d-8fa3-48d7-b7d7-5d8fc7445040 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:13:55.944864 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cccfea7d-8fa3-48d7-b7d7-5d8fc7445040 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:13:55.945338 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cccfea7d-8fa3-48d7-b7d7-5d8fc7445040 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:13:55.945757 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cccfea7d-8fa3-48d7-b7d7-5d8fc7445040 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:13:56.076901 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-cccfea7d-8fa3-48d7-b7d7-5d8fc7445040 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:13:56.084631 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-cccfea7d-8fa3-48d7-b7d7-5d8fc7445040 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1568 microversion: 2.1 time: 0.152793 Sep 04 01:13:56.085435 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 575/1121] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:13:55 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1568 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:13:57.107658 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-8d8a613e-d43d-4c6b-806c-c1a01da2c9b9 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:13:57.108288 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-8d8a613e-d43d-4c6b-806c-c1a01da2c9b9 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:13:57.114863 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8d8a613e-d43d-4c6b-806c-c1a01da2c9b9 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:13:57.115338 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8d8a613e-d43d-4c6b-806c-c1a01da2c9b9 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:13:57.115839 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8d8a613e-d43d-4c6b-806c-c1a01da2c9b9 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:13:57.116537 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8d8a613e-d43d-4c6b-806c-c1a01da2c9b9 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:13:57.116983 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8d8a613e-d43d-4c6b-806c-c1a01da2c9b9 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:13:57.117450 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8d8a613e-d43d-4c6b-806c-c1a01da2c9b9 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:13:57.253389 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-8d8a613e-d43d-4c6b-806c-c1a01da2c9b9 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:13:57.261028 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-8d8a613e-d43d-4c6b-806c-c1a01da2c9b9 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1568 microversion: 2.1 time: 0.157512 Sep 04 01:13:57.261693 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 547/1122] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:13:57 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1568 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:13:58.284978 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-e3097a46-0d15-45c0-a03b-371bfa541c67 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:13:58.285624 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-e3097a46-0d15-45c0-a03b-371bfa541c67 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:13:58.292085 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e3097a46-0d15-45c0-a03b-371bfa541c67 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:13:58.292562 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e3097a46-0d15-45c0-a03b-371bfa541c67 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:13:58.292996 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e3097a46-0d15-45c0-a03b-371bfa541c67 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:13:58.293703 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e3097a46-0d15-45c0-a03b-371bfa541c67 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:13:58.294150 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e3097a46-0d15-45c0-a03b-371bfa541c67 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:13:58.294570 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e3097a46-0d15-45c0-a03b-371bfa541c67 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:13:58.426033 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e3097a46-0d15-45c0-a03b-371bfa541c67 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:13:58.433936 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-e3097a46-0d15-45c0-a03b-371bfa541c67 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1568 microversion: 2.1 time: 0.152821 Sep 04 01:13:58.434539 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 576/1123] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:13:58 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1568 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:13:58.897341 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-31a8d5c2-f41b-4bd1-985d-bad740f5799d req-bf3f85fe-c1e5-4c50-9fa4-5a9a1f0e3d0c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "74df11dc-31bd-4745-83bc-4b3c430d02ff", "name": "network-vif-unplugged", "status": "completed", "tag": "d072bfcf-41bd-430e-800c-1d1a3b17d915"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:13:58.907623 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-31a8d5c2-f41b-4bd1-985d-bad740f5799d req-bf3f85fe-c1e5-4c50-9fa4-5a9a1f0e3d0c service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:13:58.908100 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-31a8d5c2-f41b-4bd1-985d-bad740f5799d req-bf3f85fe-c1e5-4c50-9fa4-5a9a1f0e3d0c service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:13:58.908566 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-31a8d5c2-f41b-4bd1-985d-bad740f5799d req-bf3f85fe-c1e5-4c50-9fa4-5a9a1f0e3d0c service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:13:58.920551 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-31a8d5c2-f41b-4bd1-985d-bad740f5799d req-bf3f85fe-c1e5-4c50-9fa4-5a9a1f0e3d0c service nova] Creating event network-vif-unplugged:d072bfcf-41bd-430e-800c-1d1a3b17d915 for instance 74df11dc-31bd-4745-83bc-4b3c430d02ff on hv-cinder-maste Sep 04 01:13:58.925371 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-31a8d5c2-f41b-4bd1-985d-bad740f5799d req-bf3f85fe-c1e5-4c50-9fa4-5a9a1f0e3d0c service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.031829 Sep 04 01:13:58.925944 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 548/1124] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:13:58 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:13:59.458291 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-6bfd23ee-e282-450e-801c-0de74b38049c tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:13:59.458930 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-6bfd23ee-e282-450e-801c-0de74b38049c tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:13:59.465512 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6bfd23ee-e282-450e-801c-0de74b38049c tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:13:59.465996 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6bfd23ee-e282-450e-801c-0de74b38049c tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:13:59.466433 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6bfd23ee-e282-450e-801c-0de74b38049c tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:13:59.467755 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6bfd23ee-e282-450e-801c-0de74b38049c tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:13:59.468397 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6bfd23ee-e282-450e-801c-0de74b38049c tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:13:59.468826 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-6bfd23ee-e282-450e-801c-0de74b38049c tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:13:59.598281 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-6bfd23ee-e282-450e-801c-0de74b38049c tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:13:59.606136 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-6bfd23ee-e282-450e-801c-0de74b38049c tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1568 microversion: 2.1 time: 0.151773 Sep 04 01:13:59.606698 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 577/1125] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:13:59 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1568 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:14:00.631482 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-1de27462-19c6-4658-baba-b32b4a22618f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:14:00.632185 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-1de27462-19c6-4658-baba-b32b4a22618f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:14:00.639737 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1de27462-19c6-4658-baba-b32b4a22618f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:00.640239 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1de27462-19c6-4658-baba-b32b4a22618f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:00.640678 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1de27462-19c6-4658-baba-b32b4a22618f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:00.641544 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1de27462-19c6-4658-baba-b32b4a22618f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:00.642031 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1de27462-19c6-4658-baba-b32b4a22618f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:00.642488 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1de27462-19c6-4658-baba-b32b4a22618f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:00.788135 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-1de27462-19c6-4658-baba-b32b4a22618f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:14:00.796617 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-1de27462-19c6-4658-baba-b32b4a22618f tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1568 microversion: 2.1 time: 0.170426 Sep 04 01:14:00.797318 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 549/1126] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:14:00 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1568 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:14:01.820289 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-2cfb6af6-2677-42c6-8dc3-f829b99dc402 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:14:01.821055 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-2cfb6af6-2677-42c6-8dc3-f829b99dc402 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:14:01.829206 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2cfb6af6-2677-42c6-8dc3-f829b99dc402 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:01.829206 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2cfb6af6-2677-42c6-8dc3-f829b99dc402 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:01.832508 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2cfb6af6-2677-42c6-8dc3-f829b99dc402 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:01.832508 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2cfb6af6-2677-42c6-8dc3-f829b99dc402 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:01.832508 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2cfb6af6-2677-42c6-8dc3-f829b99dc402 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:01.832508 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2cfb6af6-2677-42c6-8dc3-f829b99dc402 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:01.970358 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-2cfb6af6-2677-42c6-8dc3-f829b99dc402 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:14:01.978292 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-2cfb6af6-2677-42c6-8dc3-f829b99dc402 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1449 microversion: 2.1 time: 0.162139 Sep 04 01:14:01.978854 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 578/1127] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:14:01 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1449 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:14:02.229298 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-b67c675e-ac60-49ee-a188-545a6a795441 req-036f9fd3-097f-483b-99c0-755d1cd19fba service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "74df11dc-31bd-4745-83bc-4b3c430d02ff", "name": "network-vif-deleted", "tag": "d072bfcf-41bd-430e-800c-1d1a3b17d915"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:14:02.235062 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-b67c675e-ac60-49ee-a188-545a6a795441 req-036f9fd3-097f-483b-99c0-755d1cd19fba service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:02.235539 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-b67c675e-ac60-49ee-a188-545a6a795441 req-036f9fd3-097f-483b-99c0-755d1cd19fba service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:02.235961 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-b67c675e-ac60-49ee-a188-545a6a795441 req-036f9fd3-097f-483b-99c0-755d1cd19fba service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:02.247004 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-b67c675e-ac60-49ee-a188-545a6a795441 req-036f9fd3-097f-483b-99c0-755d1cd19fba service nova] Creating event network-vif-deleted:d072bfcf-41bd-430e-800c-1d1a3b17d915 for instance 74df11dc-31bd-4745-83bc-4b3c430d02ff on hv-cinder-maste Sep 04 01:14:02.252124 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-b67c675e-ac60-49ee-a188-545a6a795441 req-036f9fd3-097f-483b-99c0-755d1cd19fba service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030937 Sep 04 01:14:02.252819 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 550/1128] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:14:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:14:03.002355 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b125e3e9-7403-49cd-8c82-ca26083e57ee tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:14:03.006474 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-b125e3e9-7403-49cd-8c82-ca26083e57ee tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:14:03.013002 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b125e3e9-7403-49cd-8c82-ca26083e57ee tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:03.014122 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b125e3e9-7403-49cd-8c82-ca26083e57ee tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:03.014566 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b125e3e9-7403-49cd-8c82-ca26083e57ee tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:03.015409 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b125e3e9-7403-49cd-8c82-ca26083e57ee tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:03.015708 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b125e3e9-7403-49cd-8c82-ca26083e57ee tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:03.016131 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b125e3e9-7403-49cd-8c82-ca26083e57ee tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:03.081945 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-b125e3e9-7403-49cd-8c82-ca26083e57ee tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:14:03.089860 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-b125e3e9-7403-49cd-8c82-ca26083e57ee tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1218 microversion: 2.1 time: 0.091958 Sep 04 01:14:03.090424 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 579/1129] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:14:02 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1218 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:14:04.113670 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-9786eee2-421d-4509-b18a-608678f3a3d5 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:14:04.114470 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-9786eee2-421d-4509-b18a-608678f3a3d5 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:14:04.120820 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9786eee2-421d-4509-b18a-608678f3a3d5 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:04.121378 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9786eee2-421d-4509-b18a-608678f3a3d5 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:04.121803 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9786eee2-421d-4509-b18a-608678f3a3d5 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:04.122492 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9786eee2-421d-4509-b18a-608678f3a3d5 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:04.123009 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9786eee2-421d-4509-b18a-608678f3a3d5 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:04.123402 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9786eee2-421d-4509-b18a-608678f3a3d5 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:04.155544 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9786eee2-421d-4509-b18a-608678f3a3d5 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:04.156017 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9786eee2-421d-4509-b18a-608678f3a3d5 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:04.156450 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9786eee2-421d-4509-b18a-608678f3a3d5 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:04.156888 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.objects.instance [None req-9786eee2-421d-4509-b18a-608678f3a3d5 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lazy-loading 'fault' on Instance uuid 74df11dc-31bd-4745-83bc-4b3c430d02ff {{(pid=121037) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:14:04.200669 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-9786eee2-421d-4509-b18a-608678f3a3d5 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:14:04.209291 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-9786eee2-421d-4509-b18a-608678f3a3d5 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1223 microversion: 2.1 time: 0.100262 Sep 04 01:14:04.209858 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 551/1130] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:14:04 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1223 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:14:05.233635 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b795630a-e2c8-4e78-bd7c-8e1093e9c113 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:14:05.234245 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-b795630a-e2c8-4e78-bd7c-8e1093e9c113 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:14:05.240633 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b795630a-e2c8-4e78-bd7c-8e1093e9c113 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:05.241130 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b795630a-e2c8-4e78-bd7c-8e1093e9c113 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:05.241568 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b795630a-e2c8-4e78-bd7c-8e1093e9c113 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:05.242254 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b795630a-e2c8-4e78-bd7c-8e1093e9c113 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:05.242705 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b795630a-e2c8-4e78-bd7c-8e1093e9c113 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:05.243128 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b795630a-e2c8-4e78-bd7c-8e1093e9c113 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:05.275106 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b795630a-e2c8-4e78-bd7c-8e1093e9c113 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:05.275570 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b795630a-e2c8-4e78-bd7c-8e1093e9c113 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:05.275997 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b795630a-e2c8-4e78-bd7c-8e1093e9c113 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:05.276435 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.objects.instance [None req-b795630a-e2c8-4e78-bd7c-8e1093e9c113 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lazy-loading 'fault' on Instance uuid 74df11dc-31bd-4745-83bc-4b3c430d02ff {{(pid=121036) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:14:05.317363 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-b795630a-e2c8-4e78-bd7c-8e1093e9c113 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '13182cef4cf8458ab34779ffd7514c6c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '735896cd6b3b4c448dd94b3bf7577cb8', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:14:05.325046 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-b795630a-e2c8-4e78-bd7c-8e1093e9c113 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 200 len: 1223 microversion: 2.1 time: 0.095905 Sep 04 01:14:05.325643 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 580/1131] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:14:05 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 1223 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:14:06.347853 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-d964d738-4b75-41a6-a472-cb46473cc887 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:14:06.348465 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-d964d738-4b75-41a6-a472-cb46473cc887 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] [instance: 74df11dc-31bd-4745-83bc-4b3c430d02ff] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:14:06.354960 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d964d738-4b75-41a6-a472-cb46473cc887 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:06.355438 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d964d738-4b75-41a6-a472-cb46473cc887 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:06.360936 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d964d738-4b75-41a6-a472-cb46473cc887 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:06.360936 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d964d738-4b75-41a6-a472-cb46473cc887 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:06.360936 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d964d738-4b75-41a6-a472-cb46473cc887 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:06.360936 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d964d738-4b75-41a6-a472-cb46473cc887 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:06.372027 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.wsgi [None req-d964d738-4b75-41a6-a472-cb46473cc887 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] HTTP exception thrown: Instance 74df11dc-31bd-4745-83bc-4b3c430d02ff could not be found. Sep 04 01:14:06.374324 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-d964d738-4b75-41a6-a472-cb46473cc887 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Request method failure captured: Sep 04 01:14:06.374324 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: request: GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff HTTP/1.1 Sep 04 01:14:06.374324 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept: application/json Sep 04 01:14:06.374324 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept-Encoding: identity Sep 04 01:14:06.374324 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Connection: close Sep 04 01:14:06.374324 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Length: 0 Sep 04 01:14:06.374324 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Type: application/json Sep 04 01:14:06.374324 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Host: 10.222.0.42 Sep 04 01:14:06.374324 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Openstack-System-Scope: None Sep 04 01:14:06.374324 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: User-Agent: python-urllib3/1.26.12 Sep 04 01:14:06.374324 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Auth-Token: gAAAAABjE_qTTWkZkWIVV3pEI6r6MPsN82_y9-dy5d2BbCGWFNARKsxQv6jlX7t3NNUBzR8Zkz2AIkwv2lWoV7_7u3pgWS9rl6n-L54Irmk5ikzuhAIqaZVnX8kDi4SIuFjRyJcrE995ZQLlnnaUPi32WFVSer5j5sCgyCLMKv16rQkWXg8XG20 Sep 04 01:14:06.374324 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Id: None Sep 04 01:14:06.374324 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Name: None Sep 04 01:14:06.374324 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Identity-Status: Confirmed Sep 04 01:14:06.374324 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Is-Admin-Project: True Sep 04 01:14:06.374324 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Id: default Sep 04 01:14:06.374324 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Name: Default Sep 04 01:14:06.374324 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Id: 735896cd6b3b4c448dd94b3bf7577cb8 Sep 04 01:14:06.374324 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Name: tempest-TestShelveInstance-193062993 Sep 04 01:14:06.374324 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Role: reader,member Sep 04 01:14:06.374324 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Roles: reader,member Sep 04 01:14:06.376025 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/735896cd6b3b4c448dd94b3bf7577cb8"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_735896cd6b3b4c448dd94b3bf7577cb8", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/735896cd6b3b4c448dd94b3bf7577cb8"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/735896cd6b3b4c448dd94b3bf7577cb8"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:14:06.376025 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant: tempest-TestShelveInstance-193062993 Sep 04 01:14:06.376025 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Id: 735896cd6b3b4c448dd94b3bf7577cb8 Sep 04 01:14:06.376025 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Name: tempest-TestShelveInstance-193062993 Sep 04 01:14:06.376025 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User: tempest-TestShelveInstance-193062993-project Sep 04 01:14:06.376025 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Id: default Sep 04 01:14:06.376025 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Name: Default Sep 04 01:14:06.376025 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Id: 13182cef4cf8458ab34779ffd7514c6c Sep 04 01:14:06.376025 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Name: tempest-TestShelveInstance-193062993-project Sep 04 01:14:06.376025 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: method: > Sep 04 01:14:06.376025 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: exception: Instance 74df11dc-31bd-4745-83bc-4b3c430d02ff could not be found. Sep 04 01:14:06.376025 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: action_args: {'id': '74df11dc-31bd-4745-83bc-4b3c430d02ff'} Sep 04 01:14:06.377101 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:14:06.377101 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:14:06.377101 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:14:06.377101 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:14:06.377101 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:14:06.377101 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:14:06.377101 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:14:06.377101 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:14:06.377101 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:14:06.377101 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:14:06.377101 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 74df11dc-31bd-4745-83bc-4b3c430d02ff could not be found. Sep 04 01:14:06.377101 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:14:06.377101 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:14:06.377101 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:14:06.377101 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:14:06.377101 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:14:06.377101 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:14:06.377101 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:14:06.377101 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:14:06.377101 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:14:06.378506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:14:06.378506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 04 01:14:06.378506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 04 01:14:06.378506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:14:06.378506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:14:06.378506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:14:06.378506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:14:06.378506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 74df11dc-31bd-4745-83bc-4b3c430d02ff could not be found. Sep 04 01:14:06.378506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:14:06.378506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:14:06.378506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:14:06.378506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:14:06.378506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:14:06.378506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:14:06.378506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:14:06.378506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:14:06.378506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 74df11dc-31bd-4745-83bc-4b3c430d02ff could not be found. Sep 04 01:14:06.378506 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:14:06.380522 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-d964d738-4b75-41a6-a472-cb46473cc887 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Returning 404 to user: Instance 74df11dc-31bd-4745-83bc-4b3c430d02ff could not be found. {{(pid=121037) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:14:06.380522 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-d964d738-4b75-41a6-a472-cb46473cc887 tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff" status: 404 len: 111 microversion: 2.1 time: 0.031753 Sep 04 01:14:06.380522 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 552/1132] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:14:06 2022] GET /compute/v2.1/servers/74df11dc-31bd-4745-83bc-4b3c430d02ff => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 04 01:14:06.626104 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-5a7e81aa-fb36-40bf-83dc-a07663e0537c tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] Calling method '.version_select at 0x7fc10c4b01f0>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:14:06.713486 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-5a7e81aa-fb36-40bf-83dc-a07663e0537c tempest-TestShelveInstance-193062993 tempest-TestShelveInstance-193062993-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-832617863" status: 202 len: 0 microversion: 2.1 time: 0.091346 Sep 04 01:14:06.714065 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 581/1133] 10.222.0.42 () {58 vars in 1251 bytes} [Sun Sep 4 01:14:06 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-832617863 => generated 0 bytes in 93 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 04 01:14:38.556813 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-195b455b-1443-4905-a6d9-4cd4336a62cf tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Action: 'create', calling method: .version_select at 0x7fc10c8edee0>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-1802603257", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBP1fjPKP7Bj1nvRHZXD66lQqV/O2dOlAuLNv9uccU92ryMl5/PNDfVuhW47Abr5o3DGWOcIH711zBHqCkAwnBrt0ma4KvKtaasqFlYy0+HeNHDV91gzQuCLBtZc5bsWZow=="}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:14:38.567150 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_api:SELECT=20 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:14:38.568226 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=40 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:14:38.592905 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.quota [None req-195b455b-1443-4905-a6d9-4cd4336a62cf tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Getting quotas for project 027792a6118140099aaf522ca15bb07e. Resources: {'key_pairs'} {{(pid=121037) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 04 01:14:38.597564 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.quota [None req-195b455b-1443-4905-a6d9-4cd4336a62cf tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Getting quotas for user 9b5efa37b94b47949344d00ff88be4fb and project 027792a6118140099aaf522ca15bb07e. Resources: {'key_pairs'} {{(pid=121037) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 04 01:14:38.703276 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-195b455b-1443-4905-a6d9-4cd4336a62cf tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 403 microversion: 2.1 time: 0.152518 Sep 04 01:14:38.703796 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 553/1134] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Sep 4 01:14:38 2022] POST /compute/v2.1/os-keypairs => generated 403 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:14:38.827035 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=58,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:14:38.828052 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_api:SELECT=26,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:14:38.832564 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-de681008-8fe9-450f-a2ce-82e61c9b5055 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-1389319086", "imageRef": "14e5534f-1a22-44d0-9fe4-9b962cd8b9fc", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-1802603257", "networks": [{"uuid": "93fca803-def7-462d-b519-0eb07f4ac511"}]}} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:14:38.941863 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.network.neutron [None req-de681008-8fe9-450f-a2ce-82e61c9b5055 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] validate_networks() for [('93fca803-def7-462d-b519-0eb07f4ac511', None, None, None, None, None)] {{(pid=121036) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 04 01:14:39.713712 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-de681008-8fe9-450f-a2ce-82e61c9b5055 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Flavor limits 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 04 01:14:39.713712 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-de681008-8fe9-450f-a2ce-82e61c9b5055 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Image limits 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 04 01:14:39.713712 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-de681008-8fe9-450f-a2ce-82e61c9b5055 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Flavor pref 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 04 01:14:39.714144 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-de681008-8fe9-450f-a2ce-82e61c9b5055 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Image pref 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 04 01:14:39.714886 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-de681008-8fe9-450f-a2ce-82e61c9b5055 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 04 01:14:39.732621 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-de681008-8fe9-450f-a2ce-82e61c9b5055 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:39.733283 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-de681008-8fe9-450f-a2ce-82e61c9b5055 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:39.733885 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-de681008-8fe9-450f-a2ce-82e61c9b5055 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:39.734867 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-de681008-8fe9-450f-a2ce-82e61c9b5055 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:39.735487 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-de681008-8fe9-450f-a2ce-82e61c9b5055 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:39.736183 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-de681008-8fe9-450f-a2ce-82e61c9b5055 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:39.763032 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-de681008-8fe9-450f-a2ce-82e61c9b5055 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Getting quotas for project 027792a6118140099aaf522ca15bb07e. Resources: {'ram', 'instances', 'cores'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 04 01:14:39.771321 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-de681008-8fe9-450f-a2ce-82e61c9b5055 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Getting quotas for user 9b5efa37b94b47949344d00ff88be4fb and project 027792a6118140099aaf522ca15bb07e. Resources: {'ram', 'instances', 'cores'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 04 01:14:39.785044 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-de681008-8fe9-450f-a2ce-82e61c9b5055 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Going to run 1 instances... {{(pid=121036) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 04 01:14:39.787409 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-de681008-8fe9-450f-a2ce-82e61c9b5055 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='14e5534f-1a22-44d0-9fe4-9b962cd8b9fc',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=121036) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 04 01:14:39.941476 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR oslo.messaging._drivers.impl_rabbit [None req-de681008-8fe9-450f-a2ce-82e61c9b5055 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [622b2e7f-ad37-425e-be85-73ab8639dbcd] AMQP server on 10.222.0.42:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Sep 04 01:14:40.825939 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-1168196d-3a1b-4ed6-8457-50fc36f09694 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Action: 'create', calling method: .version_select at 0x7fc10c614ee0>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-436573754", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMbFVafBk3l+JlqlBJH/p4ndSWWVyN66VOeC8kQEAqRdsB/BE4OeqyTXLyo8etPn8qle0L7MSljeQ7rI0C/iZ3svxqTjS6Qlf7+IPK/DCwULwB+PmDPoYUBk+czAIa6pgw=="}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:14:40.833731 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.quota [None req-1168196d-3a1b-4ed6-8457-50fc36f09694 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Getting quotas for project 2506b96ac13541e6a7a47c5aedca67e5. Resources: {'key_pairs'} {{(pid=121037) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 04 01:14:40.837585 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.quota [None req-1168196d-3a1b-4ed6-8457-50fc36f09694 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Getting quotas for user 48c4f0c78a364c00a793666eeb5a520c and project 2506b96ac13541e6a7a47c5aedca67e5. Resources: {'key_pairs'} {{(pid=121037) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 04 01:14:40.920175 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-1168196d-3a1b-4ed6-8457-50fc36f09694 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "POST /compute/v2.1/os-keypairs" status: 200 len: 399 microversion: 2.1 time: 0.383814 Sep 04 01:14:40.921026 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 554/1135] 10.222.0.42 () {60 vars in 1159 bytes} [Sun Sep 4 01:14:40 2022] POST /compute/v2.1/os-keypairs => generated 399 bytes in 385 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:14:40.949223 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO oslo.messaging._drivers.impl_rabbit [None req-de681008-8fe9-450f-a2ce-82e61c9b5055 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [622b2e7f-ad37-425e-be85-73ab8639dbcd] Reconnected to AMQP server on 10.222.0.42:5672 via [amqp] client with port 45238. Sep 04 01:14:40.954055 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-de681008-8fe9-450f-a2ce-82e61c9b5055 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.128665 Sep 04 01:14:40.954704 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 582/1136] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 4 01:14:38 2022] POST /compute/v2.1/servers => generated 380 bytes in 2130 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 04 01:14:40.976585 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-3d9f7a9f-840d-498b-a67a-37c1b4069ebc tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:14:40.977281 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-3d9f7a9f-840d-498b-a67a-37c1b4069ebc tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:14:41.051872 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-3d9f7a9f-840d-498b-a67a-37c1b4069ebc tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:14:41.061328 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-3d9f7a9f-840d-498b-a67a-37c1b4069ebc tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1259 microversion: 2.1 time: 0.098336 Sep 04 01:14:41.062223 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 555/1137] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:14:40 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1259 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:14:42.087595 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b9d9c9d6-f9af-4ade-bfb9-e75365f41759 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:14:42.088465 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-b9d9c9d6-f9af-4ade-bfb9-e75365f41759 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:14:42.156008 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-b9d9c9d6-f9af-4ade-bfb9-e75365f41759 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:14:42.163265 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-b9d9c9d6-f9af-4ade-bfb9-e75365f41759 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1259 microversion: 2.1 time: 0.080385 Sep 04 01:14:42.163840 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 583/1138] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:14:42 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1259 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:14:43.187782 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-c2926752-f8bf-4eff-8c35-7043a22863fd tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:14:43.188561 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-c2926752-f8bf-4eff-8c35-7043a22863fd tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:14:43.196134 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c2926752-f8bf-4eff-8c35-7043a22863fd tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:43.196817 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c2926752-f8bf-4eff-8c35-7043a22863fd tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:43.197456 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c2926752-f8bf-4eff-8c35-7043a22863fd tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:43.198278 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c2926752-f8bf-4eff-8c35-7043a22863fd tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:43.198844 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c2926752-f8bf-4eff-8c35-7043a22863fd tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:43.199390 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c2926752-f8bf-4eff-8c35-7043a22863fd tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:43.285017 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-c2926752-f8bf-4eff-8c35-7043a22863fd tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:14:43.292752 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-c2926752-f8bf-4eff-8c35-7043a22863fd tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1311 microversion: 2.1 time: 0.109972 Sep 04 01:14:43.293947 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 556/1139] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:14:43 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1311 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:14:44.320119 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-49429e9d-9a32-40dd-87fb-1e5067b21dcf tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:14:44.320890 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-49429e9d-9a32-40dd-87fb-1e5067b21dcf tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:14:44.329026 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-49429e9d-9a32-40dd-87fb-1e5067b21dcf tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:44.329580 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-49429e9d-9a32-40dd-87fb-1e5067b21dcf tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:44.330219 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-49429e9d-9a32-40dd-87fb-1e5067b21dcf tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:44.331257 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-49429e9d-9a32-40dd-87fb-1e5067b21dcf tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:44.353169 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-49429e9d-9a32-40dd-87fb-1e5067b21dcf tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.022s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:44.353935 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-49429e9d-9a32-40dd-87fb-1e5067b21dcf tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:44.466642 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-49429e9d-9a32-40dd-87fb-1e5067b21dcf tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:14:44.474658 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-49429e9d-9a32-40dd-87fb-1e5067b21dcf tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1317 microversion: 2.1 time: 0.158509 Sep 04 01:14:44.475544 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 584/1140] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:14:44 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1317 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:14:45.504764 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-560d701e-6626-482f-803f-4eac0a86a673 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:14:45.506356 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-560d701e-6626-482f-803f-4eac0a86a673 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:14:45.516438 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-560d701e-6626-482f-803f-4eac0a86a673 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:45.517454 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-560d701e-6626-482f-803f-4eac0a86a673 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:45.518095 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-560d701e-6626-482f-803f-4eac0a86a673 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:45.519776 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-560d701e-6626-482f-803f-4eac0a86a673 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:45.520519 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-560d701e-6626-482f-803f-4eac0a86a673 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:45.521479 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-560d701e-6626-482f-803f-4eac0a86a673 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:45.829376 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-560d701e-6626-482f-803f-4eac0a86a673 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:14:45.841804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-560d701e-6626-482f-803f-4eac0a86a673 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1359 microversion: 2.1 time: 0.344025 Sep 04 01:14:45.847014 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 557/1141] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:14:45 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1359 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:14:45.977504 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-e44fe168-7231-4167-a31f-29626be8fa15 req-2954d0ff-5b26-445d-86ad-3d450c7d751d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64", "tag": "17937f22-477f-47ec-b6b4-0ea96fee6512"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:14:46.003727 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-e44fe168-7231-4167-a31f-29626be8fa15 req-2954d0ff-5b26-445d-86ad-3d450c7d751d service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:46.004647 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-e44fe168-7231-4167-a31f-29626be8fa15 req-2954d0ff-5b26-445d-86ad-3d450c7d751d service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:46.006237 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-e44fe168-7231-4167-a31f-29626be8fa15 req-2954d0ff-5b26-445d-86ad-3d450c7d751d service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:46.030416 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-e44fe168-7231-4167-a31f-29626be8fa15 req-2954d0ff-5b26-445d-86ad-3d450c7d751d service nova] Creating event network-changed:17937f22-477f-47ec-b6b4-0ea96fee6512 for instance 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 on hv-cinder-maste Sep 04 01:14:46.040890 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-e44fe168-7231-4167-a31f-29626be8fa15 req-2954d0ff-5b26-445d-86ad-3d450c7d751d service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.072086 Sep 04 01:14:46.043232 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 585/1142] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:14:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:14:46.876653 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-afbffdc3-4a3b-4186-a95d-864e5fc29b4c tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:14:46.878841 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-afbffdc3-4a3b-4186-a95d-864e5fc29b4c tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:14:46.892607 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-afbffdc3-4a3b-4186-a95d-864e5fc29b4c tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:46.893921 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-afbffdc3-4a3b-4186-a95d-864e5fc29b4c tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:46.896387 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-afbffdc3-4a3b-4186-a95d-864e5fc29b4c tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:46.899374 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-afbffdc3-4a3b-4186-a95d-864e5fc29b4c tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:46.900398 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-afbffdc3-4a3b-4186-a95d-864e5fc29b4c tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:46.902197 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-afbffdc3-4a3b-4186-a95d-864e5fc29b4c tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:47.097438 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-afbffdc3-4a3b-4186-a95d-864e5fc29b4c tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:14:47.107249 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-afbffdc3-4a3b-4186-a95d-864e5fc29b4c tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1359 microversion: 2.1 time: 0.236904 Sep 04 01:14:47.109037 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 558/1143] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:14:46 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1359 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:14:48.133478 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-8052c6a3-27ae-43ef-b461-c37979a4ecd2 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:14:48.134234 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-8052c6a3-27ae-43ef-b461-c37979a4ecd2 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:14:48.141179 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8052c6a3-27ae-43ef-b461-c37979a4ecd2 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:48.141576 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8052c6a3-27ae-43ef-b461-c37979a4ecd2 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:48.142002 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8052c6a3-27ae-43ef-b461-c37979a4ecd2 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:48.142688 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8052c6a3-27ae-43ef-b461-c37979a4ecd2 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:48.143129 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8052c6a3-27ae-43ef-b461-c37979a4ecd2 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:48.143544 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8052c6a3-27ae-43ef-b461-c37979a4ecd2 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:48.310565 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-8052c6a3-27ae-43ef-b461-c37979a4ecd2 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:14:48.323137 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-8052c6a3-27ae-43ef-b461-c37979a4ecd2 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1359 microversion: 2.1 time: 0.194582 Sep 04 01:14:48.324636 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 586/1144] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:14:48 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1359 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:14:49.349497 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-9a4d00c3-bbbe-4e66-be83-c566f7dc1fcc tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:14:49.351543 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-9a4d00c3-bbbe-4e66-be83-c566f7dc1fcc tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:14:49.358229 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9a4d00c3-bbbe-4e66-be83-c566f7dc1fcc tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:49.363654 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9a4d00c3-bbbe-4e66-be83-c566f7dc1fcc tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:49.363654 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9a4d00c3-bbbe-4e66-be83-c566f7dc1fcc tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:49.363654 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9a4d00c3-bbbe-4e66-be83-c566f7dc1fcc tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:49.363654 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9a4d00c3-bbbe-4e66-be83-c566f7dc1fcc tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:49.364600 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-9a4d00c3-bbbe-4e66-be83-c566f7dc1fcc tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:49.542153 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-9a4d00c3-bbbe-4e66-be83-c566f7dc1fcc tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:14:49.553563 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-9a4d00c3-bbbe-4e66-be83-c566f7dc1fcc tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1359 microversion: 2.1 time: 0.207977 Sep 04 01:14:49.555028 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 559/1145] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:14:49 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1359 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:14:50.623283 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-99d6929e-e0d4-43ca-abbf-751efec1a4c6 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:14:50.623927 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-99d6929e-e0d4-43ca-abbf-751efec1a4c6 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:14:50.626349 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell0:SELECT=2 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:14:50.631671 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-99d6929e-e0d4-43ca-abbf-751efec1a4c6 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:50.637727 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-99d6929e-e0d4-43ca-abbf-751efec1a4c6 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:50.637727 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-99d6929e-e0d4-43ca-abbf-751efec1a4c6 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:50.637727 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-99d6929e-e0d4-43ca-abbf-751efec1a4c6 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:50.637727 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-99d6929e-e0d4-43ca-abbf-751efec1a4c6 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:50.638142 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-99d6929e-e0d4-43ca-abbf-751efec1a4c6 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:50.836792 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-99d6929e-e0d4-43ca-abbf-751efec1a4c6 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:14:50.848567 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-99d6929e-e0d4-43ca-abbf-751efec1a4c6 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1359 microversion: 2.1 time: 0.229602 Sep 04 01:14:50.849530 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 587/1146] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:14:50 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1359 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:14:51.880693 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-30a2dc25-53ce-42cf-9b4a-8c4ca63521d4 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:14:51.882228 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-30a2dc25-53ce-42cf-9b4a-8c4ca63521d4 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:14:51.886551 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell0:SELECT=10 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:14:51.901398 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-30a2dc25-53ce-42cf-9b4a-8c4ca63521d4 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:51.902211 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-30a2dc25-53ce-42cf-9b4a-8c4ca63521d4 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:51.906562 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-30a2dc25-53ce-42cf-9b4a-8c4ca63521d4 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:51.907901 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-30a2dc25-53ce-42cf-9b4a-8c4ca63521d4 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:51.908809 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-30a2dc25-53ce-42cf-9b4a-8c4ca63521d4 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:51.909692 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-30a2dc25-53ce-42cf-9b4a-8c4ca63521d4 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:52.089647 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-30a2dc25-53ce-42cf-9b4a-8c4ca63521d4 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:14:52.099444 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-30a2dc25-53ce-42cf-9b4a-8c4ca63521d4 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1359 microversion: 2.1 time: 0.223074 Sep 04 01:14:52.103078 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 560/1147] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:14:51 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1359 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:14:53.128137 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-07e2d4d4-36fd-48a3-9e12-ed94601a0bef tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:14:53.128769 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-07e2d4d4-36fd-48a3-9e12-ed94601a0bef tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:14:53.130993 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell0:SELECT=8 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:14:53.138196 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-07e2d4d4-36fd-48a3-9e12-ed94601a0bef tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:53.138694 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-07e2d4d4-36fd-48a3-9e12-ed94601a0bef tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:53.139123 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-07e2d4d4-36fd-48a3-9e12-ed94601a0bef tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:53.139833 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-07e2d4d4-36fd-48a3-9e12-ed94601a0bef tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:53.140298 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-07e2d4d4-36fd-48a3-9e12-ed94601a0bef tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:53.141101 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-07e2d4d4-36fd-48a3-9e12-ed94601a0bef tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:53.284754 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-07e2d4d4-36fd-48a3-9e12-ed94601a0bef tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:14:53.292937 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-07e2d4d4-36fd-48a3-9e12-ed94601a0bef tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1359 microversion: 2.1 time: 0.168908 Sep 04 01:14:53.294325 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 588/1148] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:14:53 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1359 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:14:54.316778 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-86782fa9-1e81-45cf-9202-f44b1239c71f tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:14:54.317497 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-86782fa9-1e81-45cf-9202-f44b1239c71f tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:14:54.325470 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-86782fa9-1e81-45cf-9202-f44b1239c71f tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:54.326323 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-86782fa9-1e81-45cf-9202-f44b1239c71f tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:54.327016 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-86782fa9-1e81-45cf-9202-f44b1239c71f tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:54.327965 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-86782fa9-1e81-45cf-9202-f44b1239c71f tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:54.328875 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-86782fa9-1e81-45cf-9202-f44b1239c71f tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:54.329497 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-86782fa9-1e81-45cf-9202-f44b1239c71f tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:54.604252 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-86782fa9-1e81-45cf-9202-f44b1239c71f tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:14:54.623782 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-86782fa9-1e81-45cf-9202-f44b1239c71f tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1359 microversion: 2.1 time: 0.311225 Sep 04 01:14:54.637140 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 561/1149] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:14:54 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1359 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:14:55.648304 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-162a6877-750b-45a9-8f95-20cd891b18c1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:14:55.648964 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-162a6877-750b-45a9-8f95-20cd891b18c1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:14:55.656035 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-162a6877-750b-45a9-8f95-20cd891b18c1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:55.657941 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-162a6877-750b-45a9-8f95-20cd891b18c1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:55.657941 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-162a6877-750b-45a9-8f95-20cd891b18c1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:55.657941 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-162a6877-750b-45a9-8f95-20cd891b18c1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:55.658263 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-162a6877-750b-45a9-8f95-20cd891b18c1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:55.658696 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-162a6877-750b-45a9-8f95-20cd891b18c1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:55.951501 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-162a6877-750b-45a9-8f95-20cd891b18c1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:14:55.959466 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-162a6877-750b-45a9-8f95-20cd891b18c1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1359 microversion: 2.1 time: 0.315620 Sep 04 01:14:55.960038 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 589/1150] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:14:55 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1359 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:14:56.983601 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-ea68a9ca-6324-4283-814e-86d66b9a4a9b tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:14:56.984208 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-ea68a9ca-6324-4283-814e-86d66b9a4a9b tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:14:56.990753 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea68a9ca-6324-4283-814e-86d66b9a4a9b tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:56.991234 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea68a9ca-6324-4283-814e-86d66b9a4a9b tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:56.991669 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea68a9ca-6324-4283-814e-86d66b9a4a9b tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:56.992357 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea68a9ca-6324-4283-814e-86d66b9a4a9b tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:56.992807 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea68a9ca-6324-4283-814e-86d66b9a4a9b tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:56.993274 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea68a9ca-6324-4283-814e-86d66b9a4a9b tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:57.172018 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-ea68a9ca-6324-4283-814e-86d66b9a4a9b tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:14:57.180041 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-ea68a9ca-6324-4283-814e-86d66b9a4a9b tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1359 microversion: 2.1 time: 0.200678 Sep 04 01:14:57.180596 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 562/1151] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:14:56 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1359 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:14:58.208505 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-ce377002-abab-4d50-a386-ecf66908ce94 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:14:58.209430 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-ce377002-abab-4d50-a386-ecf66908ce94 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:14:58.217472 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ce377002-abab-4d50-a386-ecf66908ce94 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:58.218400 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ce377002-abab-4d50-a386-ecf66908ce94 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:58.218962 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ce377002-abab-4d50-a386-ecf66908ce94 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:58.219830 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ce377002-abab-4d50-a386-ecf66908ce94 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:58.220368 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ce377002-abab-4d50-a386-ecf66908ce94 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:58.220997 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ce377002-abab-4d50-a386-ecf66908ce94 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:58.372198 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ce377002-abab-4d50-a386-ecf66908ce94 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:14:58.380460 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-ce377002-abab-4d50-a386-ecf66908ce94 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1359 microversion: 2.1 time: 0.176451 Sep 04 01:14:58.381428 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 590/1152] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:14:58 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1359 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:14:59.404933 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-5399be55-41c4-4466-bd63-426e0c5de1c5 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:14:59.406212 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-5399be55-41c4-4466-bd63-426e0c5de1c5 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:14:59.412763 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5399be55-41c4-4466-bd63-426e0c5de1c5 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:59.413274 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5399be55-41c4-4466-bd63-426e0c5de1c5 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:59.413728 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5399be55-41c4-4466-bd63-426e0c5de1c5 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:59.414408 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5399be55-41c4-4466-bd63-426e0c5de1c5 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:14:59.414855 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5399be55-41c4-4466-bd63-426e0c5de1c5 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:14:59.415268 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5399be55-41c4-4466-bd63-426e0c5de1c5 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:14:59.542703 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-5399be55-41c4-4466-bd63-426e0c5de1c5 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:14:59.551271 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-5399be55-41c4-4466-bd63-426e0c5de1c5 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1359 microversion: 2.1 time: 0.150821 Sep 04 01:14:59.551840 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 563/1153] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:14:59 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1359 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:15:00.432996 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-7a5c2d3a-5d68-484d-abd8-c5229699da0b req-dcfe8e04-6503-48fd-93b3-5a2bd280a6b6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64", "name": "network-vif-plugged", "status": "completed", "tag": "17937f22-477f-47ec-b6b4-0ea96fee6512"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:15:00.443031 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-7a5c2d3a-5d68-484d-abd8-c5229699da0b req-dcfe8e04-6503-48fd-93b3-5a2bd280a6b6 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:00.443519 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-7a5c2d3a-5d68-484d-abd8-c5229699da0b req-dcfe8e04-6503-48fd-93b3-5a2bd280a6b6 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:00.443930 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-7a5c2d3a-5d68-484d-abd8-c5229699da0b req-dcfe8e04-6503-48fd-93b3-5a2bd280a6b6 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:00.455087 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-7a5c2d3a-5d68-484d-abd8-c5229699da0b req-dcfe8e04-6503-48fd-93b3-5a2bd280a6b6 service nova] Creating event network-vif-plugged:17937f22-477f-47ec-b6b4-0ea96fee6512 for instance 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 on hv-cinder-maste Sep 04 01:15:00.460234 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-7a5c2d3a-5d68-484d-abd8-c5229699da0b req-dcfe8e04-6503-48fd-93b3-5a2bd280a6b6 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030948 Sep 04 01:15:00.460811 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 591/1154] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:15:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:15:00.575476 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-b471f893-128a-498f-8107-48bccd08e5d1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:00.576239 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-b471f893-128a-498f-8107-48bccd08e5d1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:00.584464 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b471f893-128a-498f-8107-48bccd08e5d1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:00.584945 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b471f893-128a-498f-8107-48bccd08e5d1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:00.585417 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b471f893-128a-498f-8107-48bccd08e5d1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:00.586264 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b471f893-128a-498f-8107-48bccd08e5d1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:00.586710 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b471f893-128a-498f-8107-48bccd08e5d1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:00.587311 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b471f893-128a-498f-8107-48bccd08e5d1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:00.730163 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-b471f893-128a-498f-8107-48bccd08e5d1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:00.740057 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-b471f893-128a-498f-8107-48bccd08e5d1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1359 microversion: 2.1 time: 0.169864 Sep 04 01:15:00.751241 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 564/1155] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:00 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1359 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:15:00.866560 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b4ed60dd-eb2d-44b4-b36a-04b115e1ac63 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-412382515", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-436573754", "security_groups": [{"name": "tempest-secgroup-smoke-2067064205"}], "block_device_mapping_v2": [{"uuid": "bd96c9d6-6bd7-42bd-8b3d-5eebdc874f47", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "ca88993b-9210-4173-b062-c40cdf9e1da6"}]}} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:15:00.987338 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.network.neutron [None req-b4ed60dd-eb2d-44b4-b36a-04b115e1ac63 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] validate_networks() for [('ca88993b-9210-4173-b062-c40cdf9e1da6', None, None, None, None, None)] {{(pid=121036) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 04 01:15:01.363449 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-b4ed60dd-eb2d-44b4-b36a-04b115e1ac63 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Flavor limits 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 04 01:15:01.363872 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-b4ed60dd-eb2d-44b4-b36a-04b115e1ac63 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Image limits 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 04 01:15:01.364445 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-b4ed60dd-eb2d-44b4-b36a-04b115e1ac63 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Flavor pref 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 04 01:15:01.364861 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-b4ed60dd-eb2d-44b4-b36a-04b115e1ac63 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Image pref 0:0:0 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 04 01:15:01.365315 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.virt.hardware [None req-b4ed60dd-eb2d-44b4-b36a-04b115e1ac63 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121036) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 04 01:15:01.374446 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b4ed60dd-eb2d-44b4-b36a-04b115e1ac63 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:01.375141 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b4ed60dd-eb2d-44b4-b36a-04b115e1ac63 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:01.375574 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b4ed60dd-eb2d-44b4-b36a-04b115e1ac63 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:01.376301 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b4ed60dd-eb2d-44b4-b36a-04b115e1ac63 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:01.376752 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b4ed60dd-eb2d-44b4-b36a-04b115e1ac63 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:01.377204 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b4ed60dd-eb2d-44b4-b36a-04b115e1ac63 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:01.394245 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-b4ed60dd-eb2d-44b4-b36a-04b115e1ac63 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Getting quotas for project 2506b96ac13541e6a7a47c5aedca67e5. Resources: {'ram', 'instances', 'cores'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 04 01:15:01.398164 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.quota [None req-b4ed60dd-eb2d-44b4-b36a-04b115e1ac63 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Getting quotas for user 48c4f0c78a364c00a793666eeb5a520c and project 2506b96ac13541e6a7a47c5aedca67e5. Resources: {'ram', 'instances', 'cores'} {{(pid=121036) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 04 01:15:01.454469 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-b4ed60dd-eb2d-44b4-b36a-04b115e1ac63 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Going to run 1 instances... {{(pid=121036) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 04 01:15:01.456682 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-b4ed60dd-eb2d-44b4-b36a-04b115e1ac63 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 134e1ff6-ab5a-41a8-9afb-8891869494bf] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='bd96c9d6-6bd7-42bd-8b3d-5eebdc874f47',volume_size=None,volume_type=None)] {{(pid=121036) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 04 01:15:01.764699 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-0e554b63-6077-434d-acb0-bbd05ab692e8 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:01.765369 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-0e554b63-6077-434d-acb0-bbd05ab692e8 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:01.771781 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0e554b63-6077-434d-acb0-bbd05ab692e8 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:01.772266 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0e554b63-6077-434d-acb0-bbd05ab692e8 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:01.772786 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0e554b63-6077-434d-acb0-bbd05ab692e8 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:01.773529 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0e554b63-6077-434d-acb0-bbd05ab692e8 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:01.773908 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0e554b63-6077-434d-acb0-bbd05ab692e8 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:01.774324 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0e554b63-6077-434d-acb0-bbd05ab692e8 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:01.912103 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-0e554b63-6077-434d-acb0-bbd05ab692e8 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:01.919781 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-0e554b63-6077-434d-acb0-bbd05ab692e8 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1359 microversion: 2.1 time: 0.159873 Sep 04 01:15:01.920336 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 565/1156] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:01 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1359 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:15:01.995433 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-b4ed60dd-eb2d-44b4-b36a-04b115e1ac63 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.133532 Sep 04 01:15:01.996263 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 592/1157] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 4 01:15:00 2022] POST /compute/v2.1/servers => generated 406 bytes in 1135 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 04 01:15:02.009897 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-60be8b58-d1c9-44b6-80ce-0981efd12d30 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:02.009897 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-60be8b58-d1c9-44b6-80ce-0981efd12d30 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 134e1ff6-ab5a-41a8-9afb-8891869494bf] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:02.061345 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-60be8b58-d1c9-44b6-80ce-0981efd12d30 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:02.068140 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-60be8b58-d1c9-44b6-80ce-0981efd12d30 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf" status: 200 len: 1093 microversion: 2.1 time: 0.064744 Sep 04 01:15:02.110492 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 566/1158] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:02 2022] GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf => generated 1093 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:15:02.943200 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-829979f0-5c26-41b7-ad7a-f921012d4c61 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:02.943817 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-829979f0-5c26-41b7-ad7a-f921012d4c61 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:02.950727 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-829979f0-5c26-41b7-ad7a-f921012d4c61 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:02.953592 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-829979f0-5c26-41b7-ad7a-f921012d4c61 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:02.953592 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-829979f0-5c26-41b7-ad7a-f921012d4c61 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:02.953592 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-829979f0-5c26-41b7-ad7a-f921012d4c61 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:02.953592 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-829979f0-5c26-41b7-ad7a-f921012d4c61 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:02.954133 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-829979f0-5c26-41b7-ad7a-f921012d4c61 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:03.087536 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-829979f0-5c26-41b7-ad7a-f921012d4c61 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:03.090903 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-8b0f560b-6eb0-4845-86fc-8b4ad476773d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:03.091497 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-8b0f560b-6eb0-4845-86fc-8b4ad476773d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 134e1ff6-ab5a-41a8-9afb-8891869494bf] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:03.098021 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-829979f0-5c26-41b7-ad7a-f921012d4c61 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1549 microversion: 2.1 time: 0.159124 Sep 04 01:15:03.113451 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 593/1159] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:02 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1549 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:15:03.121573 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-fa4387db-22f8-4591-bcfb-c2aa43e117b1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:03.122188 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-fa4387db-22f8-4591-bcfb-c2aa43e117b1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:03.128375 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fa4387db-22f8-4591-bcfb-c2aa43e117b1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:03.132503 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fa4387db-22f8-4591-bcfb-c2aa43e117b1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:03.132503 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fa4387db-22f8-4591-bcfb-c2aa43e117b1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:03.132503 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fa4387db-22f8-4591-bcfb-c2aa43e117b1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:03.132503 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fa4387db-22f8-4591-bcfb-c2aa43e117b1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:03.132923 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-fa4387db-22f8-4591-bcfb-c2aa43e117b1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:03.147237 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-8b0f560b-6eb0-4845-86fc-8b4ad476773d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:03.154451 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-8b0f560b-6eb0-4845-86fc-8b4ad476773d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf" status: 200 len: 1093 microversion: 2.1 time: 0.067276 Sep 04 01:15:03.155020 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 567/1160] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:03 2022] GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf => generated 1093 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:15:03.267253 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-fa4387db-22f8-4591-bcfb-c2aa43e117b1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:03.274942 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-fa4387db-22f8-4591-bcfb-c2aa43e117b1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1549 microversion: 2.1 time: 0.157459 Sep 04 01:15:03.275536 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 594/1161] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:03 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1549 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:15:03.299966 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-17a9d00e-710f-4eef-88be-6b324ecc29c5 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:03.307795 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-17a9d00e-710f-4eef-88be-6b324ecc29c5 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:03.308567 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-17a9d00e-710f-4eef-88be-6b324ecc29c5 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Searching by: {'deleted': False, 'project_id': '027792a6118140099aaf522ca15bb07e'} {{(pid=121037) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 04 01:15:03.316067 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-17a9d00e-710f-4eef-88be-6b324ecc29c5 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:03.318202 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-17a9d00e-710f-4eef-88be-6b324ecc29c5 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:03.318202 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-17a9d00e-710f-4eef-88be-6b324ecc29c5 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:03.318202 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-17a9d00e-710f-4eef-88be-6b324ecc29c5 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:03.318202 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-17a9d00e-710f-4eef-88be-6b324ecc29c5 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:03.318598 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-17a9d00e-710f-4eef-88be-6b324ecc29c5 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:03.408660 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.multi_cell_list [None req-17a9d00e-710f-4eef-88be-6b324ecc29c5 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=121037) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 04 01:15:03.508842 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-17a9d00e-710f-4eef-88be-6b324ecc29c5 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers" status: 200 len: 343 microversion: 2.1 time: 0.214410 Sep 04 01:15:03.509465 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 568/1162] 10.222.0.42 () {58 vars in 1125 bytes} [Sun Sep 4 01:15:03 2022] GET /compute/v2.1/servers => generated 343 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:15:03.525919 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-2827b13d-da8b-427e-84cc-754dd9ab9331 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:03.525919 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-2827b13d-da8b-427e-84cc-754dd9ab9331 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:03.529330 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2827b13d-da8b-427e-84cc-754dd9ab9331 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:03.529815 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2827b13d-da8b-427e-84cc-754dd9ab9331 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:03.530254 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2827b13d-da8b-427e-84cc-754dd9ab9331 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:03.531549 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2827b13d-da8b-427e-84cc-754dd9ab9331 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:03.532004 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2827b13d-da8b-427e-84cc-754dd9ab9331 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:03.532489 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2827b13d-da8b-427e-84cc-754dd9ab9331 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:03.666530 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-2827b13d-da8b-427e-84cc-754dd9ab9331 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:03.674422 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-2827b13d-da8b-427e-84cc-754dd9ab9331 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1549 microversion: 2.1 time: 0.156233 Sep 04 01:15:03.674983 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 595/1163] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:03 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1549 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:15:04.180954 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-5ae699a5-a5ca-4aeb-b4d1-30a1e135ab30 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:04.181623 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-5ae699a5-a5ca-4aeb-b4d1-30a1e135ab30 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 134e1ff6-ab5a-41a8-9afb-8891869494bf] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:04.234693 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-5ae699a5-a5ca-4aeb-b4d1-30a1e135ab30 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:04.241757 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-5ae699a5-a5ca-4aeb-b4d1-30a1e135ab30 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf" status: 200 len: 1093 microversion: 2.1 time: 0.064659 Sep 04 01:15:04.242315 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 569/1164] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:04 2022] GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf => generated 1093 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:15:05.263884 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-5cf06131-b697-4e02-8f0c-177ea33b8b1f tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:05.264553 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-5cf06131-b697-4e02-8f0c-177ea33b8b1f tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 134e1ff6-ab5a-41a8-9afb-8891869494bf] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:05.319055 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-5cf06131-b697-4e02-8f0c-177ea33b8b1f tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:05.325926 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-5cf06131-b697-4e02-8f0c-177ea33b8b1f tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf" status: 200 len: 1093 microversion: 2.1 time: 0.065889 Sep 04 01:15:05.326481 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 596/1165] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:05 2022] GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf => generated 1093 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:15:05.646137 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-a45b0f68-b102-4c41-8911-fe4527046905 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "c0024b07-c22a-4a55-95ca-a833025840e2"}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:15:05.648418 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-a45b0f68-b102-4c41-8911-fe4527046905 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:05.655292 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a45b0f68-b102-4c41-8911-fe4527046905 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:05.655760 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a45b0f68-b102-4c41-8911-fe4527046905 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:05.656410 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a45b0f68-b102-4c41-8911-fe4527046905 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:05.657123 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a45b0f68-b102-4c41-8911-fe4527046905 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:05.657661 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a45b0f68-b102-4c41-8911-fe4527046905 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:05.658083 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a45b0f68-b102-4c41-8911-fe4527046905 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:06.348407 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-202ea9cc-7298-476c-85c3-5a1900f0d720 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:06.349014 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-202ea9cc-7298-476c-85c3-5a1900f0d720 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 134e1ff6-ab5a-41a8-9afb-8891869494bf] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:06.356343 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-202ea9cc-7298-476c-85c3-5a1900f0d720 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:06.356824 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-202ea9cc-7298-476c-85c3-5a1900f0d720 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:06.357299 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-202ea9cc-7298-476c-85c3-5a1900f0d720 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:06.357989 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-202ea9cc-7298-476c-85c3-5a1900f0d720 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:06.358440 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-202ea9cc-7298-476c-85c3-5a1900f0d720 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:06.358896 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-202ea9cc-7298-476c-85c3-5a1900f0d720 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:06.419555 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-202ea9cc-7298-476c-85c3-5a1900f0d720 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:06.427838 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-202ea9cc-7298-476c-85c3-5a1900f0d720 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf" status: 200 len: 1143 microversion: 2.1 time: 0.082920 Sep 04 01:15:06.428030 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 597/1166] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:06 2022] GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf => generated 1143 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:15:06.430536 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-a45b0f68-b102-4c41-8911-fe4527046905 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "POST /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.788204 Sep 04 01:15:06.431377 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 570/1167] 10.222.0.42 () {60 vars in 1323 bytes} [Sun Sep 4 01:15:05 2022] POST /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64/os-volume_attachments => generated 194 bytes in 791 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:15:07.450520 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-ed92d566-af6a-417b-92fe-ed50afaab2d8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:07.451207 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-ed92d566-af6a-417b-92fe-ed50afaab2d8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 134e1ff6-ab5a-41a8-9afb-8891869494bf] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:07.458358 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ed92d566-af6a-417b-92fe-ed50afaab2d8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:07.458858 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ed92d566-af6a-417b-92fe-ed50afaab2d8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:07.459296 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ed92d566-af6a-417b-92fe-ed50afaab2d8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:07.460003 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ed92d566-af6a-417b-92fe-ed50afaab2d8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:07.460454 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ed92d566-af6a-417b-92fe-ed50afaab2d8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:07.460879 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ed92d566-af6a-417b-92fe-ed50afaab2d8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:07.527821 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ed92d566-af6a-417b-92fe-ed50afaab2d8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:07.537543 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-ed92d566-af6a-417b-92fe-ed50afaab2d8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf" status: 200 len: 1135 microversion: 2.1 time: 0.090953 Sep 04 01:15:07.547701 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 598/1168] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:07 2022] GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf => generated 1135 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:15:08.562163 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-8a9bd444-b0af-4217-a1b7-ecb5c17f5518 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:08.563475 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-8a9bd444-b0af-4217-a1b7-ecb5c17f5518 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 134e1ff6-ab5a-41a8-9afb-8891869494bf] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:08.572138 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8a9bd444-b0af-4217-a1b7-ecb5c17f5518 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:08.572777 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8a9bd444-b0af-4217-a1b7-ecb5c17f5518 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:08.573876 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8a9bd444-b0af-4217-a1b7-ecb5c17f5518 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:08.575310 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8a9bd444-b0af-4217-a1b7-ecb5c17f5518 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:08.575766 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8a9bd444-b0af-4217-a1b7-ecb5c17f5518 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:08.576282 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8a9bd444-b0af-4217-a1b7-ecb5c17f5518 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:08.666345 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-8a9bd444-b0af-4217-a1b7-ecb5c17f5518 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:08.669022 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=84,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:15:08.677148 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-8a9bd444-b0af-4217-a1b7-ecb5c17f5518 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf" status: 200 len: 1191 microversion: 2.1 time: 0.118888 Sep 04 01:15:08.677847 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 571/1169] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:08 2022] GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf => generated 1191 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:15:09.701198 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-23960a06-a4ef-4243-88e2-49600bcb307b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:09.701828 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-23960a06-a4ef-4243-88e2-49600bcb307b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 134e1ff6-ab5a-41a8-9afb-8891869494bf] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:09.709986 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-23960a06-a4ef-4243-88e2-49600bcb307b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:09.710714 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-23960a06-a4ef-4243-88e2-49600bcb307b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:09.711314 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-23960a06-a4ef-4243-88e2-49600bcb307b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:09.712454 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-23960a06-a4ef-4243-88e2-49600bcb307b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:09.713096 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-23960a06-a4ef-4243-88e2-49600bcb307b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:09.713847 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-23960a06-a4ef-4243-88e2-49600bcb307b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:09.870726 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-23960a06-a4ef-4243-88e2-49600bcb307b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:09.880466 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-23960a06-a4ef-4243-88e2-49600bcb307b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf" status: 200 len: 1277 microversion: 2.1 time: 0.183341 Sep 04 01:15:09.881172 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 599/1170] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:09 2022] GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf => generated 1277 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:15:10.189342 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-6dde5ff1-a12a-4760-99dc-cd22a2d576f4 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:10.189966 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-6dde5ff1-a12a-4760-99dc-cd22a2d576f4 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:10.193895 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_api:SELECT=65,nova_api:INSERT=2 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:15:10.199027 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6dde5ff1-a12a-4760-99dc-cd22a2d576f4 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:10.199588 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6dde5ff1-a12a-4760-99dc-cd22a2d576f4 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:10.200030 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6dde5ff1-a12a-4760-99dc-cd22a2d576f4 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:10.200772 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6dde5ff1-a12a-4760-99dc-cd22a2d576f4 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:10.201259 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6dde5ff1-a12a-4760-99dc-cd22a2d576f4 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:10.201690 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-6dde5ff1-a12a-4760-99dc-cd22a2d576f4 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:10.332665 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-6dde5ff1-a12a-4760-99dc-cd22a2d576f4 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:10.340948 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-6dde5ff1-a12a-4760-99dc-cd22a2d576f4 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1595 microversion: 2.1 time: 0.156321 Sep 04 01:15:10.341559 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 572/1171] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:10 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1595 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:15:10.670521 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-da6856ff-0644-4d4b-954d-aa2edd0d6930 req-8f618d41-5273-41f9-b83e-009aac561474 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "134e1ff6-ab5a-41a8-9afb-8891869494bf", "tag": "8ad76ee4-45f9-4f07-93dd-75ebf95bbe32"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:15:10.683144 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_api:SELECT=67,nova_api:INSERT=6 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:15:10.696746 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-da6856ff-0644-4d4b-954d-aa2edd0d6930 req-8f618d41-5273-41f9-b83e-009aac561474 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:10.697405 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-da6856ff-0644-4d4b-954d-aa2edd0d6930 req-8f618d41-5273-41f9-b83e-009aac561474 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:10.698046 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-da6856ff-0644-4d4b-954d-aa2edd0d6930 req-8f618d41-5273-41f9-b83e-009aac561474 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:10.702258 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=81 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:15:10.717497 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-da6856ff-0644-4d4b-954d-aa2edd0d6930 req-8f618d41-5273-41f9-b83e-009aac561474 service nova] Creating event network-changed:8ad76ee4-45f9-4f07-93dd-75ebf95bbe32 for instance 134e1ff6-ab5a-41a8-9afb-8891869494bf on hv-cinder-maste Sep 04 01:15:10.722955 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-da6856ff-0644-4d4b-954d-aa2edd0d6930 req-8f618d41-5273-41f9-b83e-009aac561474 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.057706 Sep 04 01:15:10.723569 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 600/1172] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:15:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:15:10.903875 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-7acbb470-7c57-44ba-abf9-75b20fad1473 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:10.904362 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-7acbb470-7c57-44ba-abf9-75b20fad1473 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 134e1ff6-ab5a-41a8-9afb-8891869494bf] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:10.912476 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7acbb470-7c57-44ba-abf9-75b20fad1473 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:10.912962 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7acbb470-7c57-44ba-abf9-75b20fad1473 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:10.913619 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7acbb470-7c57-44ba-abf9-75b20fad1473 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:10.914320 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7acbb470-7c57-44ba-abf9-75b20fad1473 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:10.914983 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7acbb470-7c57-44ba-abf9-75b20fad1473 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:10.915407 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7acbb470-7c57-44ba-abf9-75b20fad1473 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:11.128716 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-7acbb470-7c57-44ba-abf9-75b20fad1473 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:11.139132 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-7acbb470-7c57-44ba-abf9-75b20fad1473 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf" status: 200 len: 1265 microversion: 2.1 time: 0.239069 Sep 04 01:15:11.139942 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 573/1173] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:10 2022] GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf => generated 1265 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:15:12.167808 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b4d20fa2-72af-415f-a677-6e3a2c98664d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:12.169109 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-b4d20fa2-72af-415f-a677-6e3a2c98664d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 134e1ff6-ab5a-41a8-9afb-8891869494bf] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:12.174833 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell0:SELECT=2 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:15:12.180025 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b4d20fa2-72af-415f-a677-6e3a2c98664d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:12.180875 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b4d20fa2-72af-415f-a677-6e3a2c98664d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:12.181517 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b4d20fa2-72af-415f-a677-6e3a2c98664d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:12.182646 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b4d20fa2-72af-415f-a677-6e3a2c98664d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:12.183258 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b4d20fa2-72af-415f-a677-6e3a2c98664d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:12.183902 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b4d20fa2-72af-415f-a677-6e3a2c98664d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:12.435842 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-b4d20fa2-72af-415f-a677-6e3a2c98664d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:12.447509 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-b4d20fa2-72af-415f-a677-6e3a2c98664d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf" status: 200 len: 1265 microversion: 2.1 time: 0.287409 Sep 04 01:15:12.448083 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 601/1174] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:12 2022] GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf => generated 1265 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:15:12.938796 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-46af918e-0bff-42c5-a13b-0aa2e7f5cfbc req-76de3994-50ba-48d8-b4e0-14ae8c085483 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64", "tag": "17937f22-477f-47ec-b6b4-0ea96fee6512"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:15:12.955993 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-46af918e-0bff-42c5-a13b-0aa2e7f5cfbc req-76de3994-50ba-48d8-b4e0-14ae8c085483 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:12.956493 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-46af918e-0bff-42c5-a13b-0aa2e7f5cfbc req-76de3994-50ba-48d8-b4e0-14ae8c085483 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:12.962410 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-46af918e-0bff-42c5-a13b-0aa2e7f5cfbc req-76de3994-50ba-48d8-b4e0-14ae8c085483 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:12.969947 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-46af918e-0bff-42c5-a13b-0aa2e7f5cfbc req-76de3994-50ba-48d8-b4e0-14ae8c085483 service nova] Creating event network-changed:17937f22-477f-47ec-b6b4-0ea96fee6512 for instance 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 on hv-cinder-maste Sep 04 01:15:12.975474 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-46af918e-0bff-42c5-a13b-0aa2e7f5cfbc req-76de3994-50ba-48d8-b4e0-14ae8c085483 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040971 Sep 04 01:15:12.976444 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 574/1175] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:15:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:15:13.471223 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-8f97a2e5-9cc0-44f7-9348-80a826afea7f tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:13.472471 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-8f97a2e5-9cc0-44f7-9348-80a826afea7f tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 134e1ff6-ab5a-41a8-9afb-8891869494bf] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:13.479096 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8f97a2e5-9cc0-44f7-9348-80a826afea7f tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:13.479694 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8f97a2e5-9cc0-44f7-9348-80a826afea7f tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:13.483260 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8f97a2e5-9cc0-44f7-9348-80a826afea7f tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:13.483260 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8f97a2e5-9cc0-44f7-9348-80a826afea7f tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:13.483260 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8f97a2e5-9cc0-44f7-9348-80a826afea7f tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:13.483260 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-8f97a2e5-9cc0-44f7-9348-80a826afea7f tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:13.649979 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-8f97a2e5-9cc0-44f7-9348-80a826afea7f tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:13.662513 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-8f97a2e5-9cc0-44f7-9348-80a826afea7f tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf" status: 200 len: 1265 microversion: 2.1 time: 0.195776 Sep 04 01:15:13.677573 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 602/1176] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:13 2022] GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf => generated 1265 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:15:14.687064 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-a1a8b07e-d24e-4703-b95a-25459fc12ba1 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:14.688515 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-a1a8b07e-d24e-4703-b95a-25459fc12ba1 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 134e1ff6-ab5a-41a8-9afb-8891869494bf] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:14.691716 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell0:SELECT=2 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:15:14.692812 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell0:SELECT=6 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:15:14.696645 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a1a8b07e-d24e-4703-b95a-25459fc12ba1 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:14.697179 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a1a8b07e-d24e-4703-b95a-25459fc12ba1 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:14.697606 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a1a8b07e-d24e-4703-b95a-25459fc12ba1 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:14.698344 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a1a8b07e-d24e-4703-b95a-25459fc12ba1 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:14.698819 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a1a8b07e-d24e-4703-b95a-25459fc12ba1 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:14.699239 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-a1a8b07e-d24e-4703-b95a-25459fc12ba1 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:14.837162 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-a1a8b07e-d24e-4703-b95a-25459fc12ba1 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:14.845908 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-a1a8b07e-d24e-4703-b95a-25459fc12ba1 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf" status: 200 len: 1265 microversion: 2.1 time: 0.163183 Sep 04 01:15:14.846493 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 575/1177] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:14 2022] GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf => generated 1265 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:15:15.836949 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-2846478d-0e6b-4b17-863b-05fb795cda9c tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-264434114"}} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:15:15.837522 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-2846478d-0e6b-4b17-863b-05fb795cda9c tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:15.839868 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell0:SELECT=8 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:15:15.844676 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2846478d-0e6b-4b17-863b-05fb795cda9c tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:15.845207 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2846478d-0e6b-4b17-863b-05fb795cda9c tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:15.845951 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2846478d-0e6b-4b17-863b-05fb795cda9c tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:15.846335 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2846478d-0e6b-4b17-863b-05fb795cda9c tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:15.846786 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2846478d-0e6b-4b17-863b-05fb795cda9c tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:15.847214 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2846478d-0e6b-4b17-863b-05fb795cda9c tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:15.870120 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-1f1cab10-7ee5-42ee-a5ad-6ddf45f7b152 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:15.882527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-1f1cab10-7ee5-42ee-a5ad-6ddf45f7b152 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 134e1ff6-ab5a-41a8-9afb-8891869494bf] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:15.882527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1f1cab10-7ee5-42ee-a5ad-6ddf45f7b152 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:15.882527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1f1cab10-7ee5-42ee-a5ad-6ddf45f7b152 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:15.882527 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1f1cab10-7ee5-42ee-a5ad-6ddf45f7b152 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:15.883002 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1f1cab10-7ee5-42ee-a5ad-6ddf45f7b152 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:15.883002 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1f1cab10-7ee5-42ee-a5ad-6ddf45f7b152 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:15.883002 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1f1cab10-7ee5-42ee-a5ad-6ddf45f7b152 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:15.996597 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.network.security_group_api [None req-2846478d-0e6b-4b17-863b-05fb795cda9c tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Adding security group 453a92ae-fa58-40a9-8940-f7ae3987450a to port 17937f22-477f-47ec-b6b4-0ea96fee6512 Sep 04 01:15:16.033700 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-1f1cab10-7ee5-42ee-a5ad-6ddf45f7b152 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:16.043916 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-1f1cab10-7ee5-42ee-a5ad-6ddf45f7b152 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf" status: 200 len: 1265 microversion: 2.1 time: 0.178441 Sep 04 01:15:16.044615 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 576/1178] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:15 2022] GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf => generated 1265 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:15:16.383539 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-2846478d-0e6b-4b17-863b-05fb795cda9c tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "POST /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64/action" status: 202 len: 0 microversion: 2.1 time: 0.550559 Sep 04 01:15:16.385890 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 603/1179] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Sep 4 01:15:15 2022] POST /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64/action => generated 0 bytes in 554 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 04 01:15:16.393515 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-a6217c4f-7f34-43ec-bb59-38ddde1e8f4e req-85adfe5b-c85b-4ec3-8d01-3af17a51a56b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64", "tag": "17937f22-477f-47ec-b6b4-0ea96fee6512"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:15:16.395454 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-e0339613-a9f7-4446-bc75-4d61efa79fb1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:16.396345 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-e0339613-a9f7-4446-bc75-4d61efa79fb1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:16.404433 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e0339613-a9f7-4446-bc75-4d61efa79fb1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:16.406017 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e0339613-a9f7-4446-bc75-4d61efa79fb1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:16.407026 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e0339613-a9f7-4446-bc75-4d61efa79fb1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:16.409343 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e0339613-a9f7-4446-bc75-4d61efa79fb1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:16.410131 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e0339613-a9f7-4446-bc75-4d61efa79fb1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:16.410796 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e0339613-a9f7-4446-bc75-4d61efa79fb1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:16.419265 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-a6217c4f-7f34-43ec-bb59-38ddde1e8f4e req-85adfe5b-c85b-4ec3-8d01-3af17a51a56b service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:16.419750 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-a6217c4f-7f34-43ec-bb59-38ddde1e8f4e req-85adfe5b-c85b-4ec3-8d01-3af17a51a56b service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:16.420185 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-a6217c4f-7f34-43ec-bb59-38ddde1e8f4e req-85adfe5b-c85b-4ec3-8d01-3af17a51a56b service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:16.445357 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-a6217c4f-7f34-43ec-bb59-38ddde1e8f4e req-85adfe5b-c85b-4ec3-8d01-3af17a51a56b service nova] Creating event network-changed:17937f22-477f-47ec-b6b4-0ea96fee6512 for instance 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 on hv-cinder-maste Sep 04 01:15:16.451311 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-a6217c4f-7f34-43ec-bb59-38ddde1e8f4e req-85adfe5b-c85b-4ec3-8d01-3af17a51a56b service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.062089 Sep 04 01:15:16.458784 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 577/1180] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:15:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:15:16.588145 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e0339613-a9f7-4446-bc75-4d61efa79fb1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:16.603301 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-e0339613-a9f7-4446-bc75-4d61efa79fb1 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1760 microversion: 2.1 time: 0.212530 Sep 04 01:15:16.603908 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 604/1181] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:16 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1760 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:15:17.070110 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-1283619d-d1af-47d1-8088-4e11a971200b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:17.071512 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-1283619d-d1af-47d1-8088-4e11a971200b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 134e1ff6-ab5a-41a8-9afb-8891869494bf] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:17.078024 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1283619d-d1af-47d1-8088-4e11a971200b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:17.080388 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1283619d-d1af-47d1-8088-4e11a971200b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:17.080388 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1283619d-d1af-47d1-8088-4e11a971200b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:17.080625 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1283619d-d1af-47d1-8088-4e11a971200b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:17.081212 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1283619d-d1af-47d1-8088-4e11a971200b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:17.081704 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1283619d-d1af-47d1-8088-4e11a971200b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:17.270217 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-1283619d-d1af-47d1-8088-4e11a971200b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:17.278185 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-1283619d-d1af-47d1-8088-4e11a971200b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf" status: 200 len: 1265 microversion: 2.1 time: 0.212906 Sep 04 01:15:17.278765 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 578/1182] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:17 2022] GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf => generated 1265 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:15:18.304730 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-92b20e4d-3ed0-4035-921c-7fc705d778b3 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:18.305474 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-92b20e4d-3ed0-4035-921c-7fc705d778b3 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 134e1ff6-ab5a-41a8-9afb-8891869494bf] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:18.313191 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-92b20e4d-3ed0-4035-921c-7fc705d778b3 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:18.314239 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-92b20e4d-3ed0-4035-921c-7fc705d778b3 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:18.315687 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-92b20e4d-3ed0-4035-921c-7fc705d778b3 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:18.316850 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-92b20e4d-3ed0-4035-921c-7fc705d778b3 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:18.317837 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-92b20e4d-3ed0-4035-921c-7fc705d778b3 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:18.318494 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-92b20e4d-3ed0-4035-921c-7fc705d778b3 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:18.486164 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-92b20e4d-3ed0-4035-921c-7fc705d778b3 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:18.496705 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-92b20e4d-3ed0-4035-921c-7fc705d778b3 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf" status: 200 len: 1265 microversion: 2.1 time: 0.198370 Sep 04 01:15:18.498256 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 605/1183] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:18 2022] GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf => generated 1265 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:15:19.521162 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-beb9aef8-82e6-4811-8dc9-22cd20d895d8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:19.521779 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-beb9aef8-82e6-4811-8dc9-22cd20d895d8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 134e1ff6-ab5a-41a8-9afb-8891869494bf] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:19.528212 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-beb9aef8-82e6-4811-8dc9-22cd20d895d8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:19.528688 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-beb9aef8-82e6-4811-8dc9-22cd20d895d8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:19.529137 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-beb9aef8-82e6-4811-8dc9-22cd20d895d8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:19.529863 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-beb9aef8-82e6-4811-8dc9-22cd20d895d8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:19.530304 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-beb9aef8-82e6-4811-8dc9-22cd20d895d8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:19.530869 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-beb9aef8-82e6-4811-8dc9-22cd20d895d8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:19.669472 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-beb9aef8-82e6-4811-8dc9-22cd20d895d8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:19.680788 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-beb9aef8-82e6-4811-8dc9-22cd20d895d8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf" status: 200 len: 1265 microversion: 2.1 time: 0.164486 Sep 04 01:15:19.682025 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 579/1184] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:19 2022] GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf => generated 1265 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:15:20.501475 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-68e5af8c-0db3-4678-88de-0cd8c47fe483 req-ed49b18c-286b-4efc-b63e-e478fd03ea17 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "134e1ff6-ab5a-41a8-9afb-8891869494bf", "name": "network-vif-plugged", "status": "completed", "tag": "8ad76ee4-45f9-4f07-93dd-75ebf95bbe32"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:15:20.513642 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-68e5af8c-0db3-4678-88de-0cd8c47fe483 req-ed49b18c-286b-4efc-b63e-e478fd03ea17 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:20.514128 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-68e5af8c-0db3-4678-88de-0cd8c47fe483 req-ed49b18c-286b-4efc-b63e-e478fd03ea17 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:20.514560 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-68e5af8c-0db3-4678-88de-0cd8c47fe483 req-ed49b18c-286b-4efc-b63e-e478fd03ea17 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:20.527648 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-68e5af8c-0db3-4678-88de-0cd8c47fe483 req-ed49b18c-286b-4efc-b63e-e478fd03ea17 service nova] Creating event network-vif-plugged:8ad76ee4-45f9-4f07-93dd-75ebf95bbe32 for instance 134e1ff6-ab5a-41a8-9afb-8891869494bf on hv-cinder-maste Sep 04 01:15:20.533462 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-68e5af8c-0db3-4678-88de-0cd8c47fe483 req-ed49b18c-286b-4efc-b63e-e478fd03ea17 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037283 Sep 04 01:15:20.538675 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 606/1185] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:15:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:15:20.703802 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-55aabaff-0855-4215-9d96-1b74a7bf359d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:20.705826 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-55aabaff-0855-4215-9d96-1b74a7bf359d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 134e1ff6-ab5a-41a8-9afb-8891869494bf] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:20.712659 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-55aabaff-0855-4215-9d96-1b74a7bf359d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:20.713187 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-55aabaff-0855-4215-9d96-1b74a7bf359d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:20.713619 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-55aabaff-0855-4215-9d96-1b74a7bf359d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:20.714308 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-55aabaff-0855-4215-9d96-1b74a7bf359d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:20.714748 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-55aabaff-0855-4215-9d96-1b74a7bf359d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:20.715171 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-55aabaff-0855-4215-9d96-1b74a7bf359d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:20.858023 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-55aabaff-0855-4215-9d96-1b74a7bf359d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:20.866949 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-55aabaff-0855-4215-9d96-1b74a7bf359d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf" status: 200 len: 1265 microversion: 2.1 time: 0.166955 Sep 04 01:15:20.867729 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 580/1186] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:20 2022] GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf => generated 1265 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:15:21.892905 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-d474c259-dc1d-4278-9c04-086f2a5dfa9e tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:21.893616 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-d474c259-dc1d-4278-9c04-086f2a5dfa9e tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 134e1ff6-ab5a-41a8-9afb-8891869494bf] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:21.899912 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d474c259-dc1d-4278-9c04-086f2a5dfa9e tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:21.900369 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d474c259-dc1d-4278-9c04-086f2a5dfa9e tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:21.900779 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d474c259-dc1d-4278-9c04-086f2a5dfa9e tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:21.901484 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d474c259-dc1d-4278-9c04-086f2a5dfa9e tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:21.901933 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d474c259-dc1d-4278-9c04-086f2a5dfa9e tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:21.902354 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d474c259-dc1d-4278-9c04-086f2a5dfa9e tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:22.069472 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-d474c259-dc1d-4278-9c04-086f2a5dfa9e tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:22.080299 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-d474c259-dc1d-4278-9c04-086f2a5dfa9e tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf" status: 200 len: 1265 microversion: 2.1 time: 0.192710 Sep 04 01:15:22.080985 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 607/1187] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:21 2022] GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf => generated 1265 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:15:23.103087 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-44fc9f2d-a182-4360-b8a7-43bbbce50fef tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:23.103668 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-44fc9f2d-a182-4360-b8a7-43bbbce50fef tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 134e1ff6-ab5a-41a8-9afb-8891869494bf] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:23.111116 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-44fc9f2d-a182-4360-b8a7-43bbbce50fef tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:23.111619 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-44fc9f2d-a182-4360-b8a7-43bbbce50fef tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:23.112056 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-44fc9f2d-a182-4360-b8a7-43bbbce50fef tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:23.112804 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-44fc9f2d-a182-4360-b8a7-43bbbce50fef tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:23.113300 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-44fc9f2d-a182-4360-b8a7-43bbbce50fef tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:23.113730 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-44fc9f2d-a182-4360-b8a7-43bbbce50fef tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:23.246890 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-44fc9f2d-a182-4360-b8a7-43bbbce50fef tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:23.255260 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-44fc9f2d-a182-4360-b8a7-43bbbce50fef tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf" status: 200 len: 1451 microversion: 2.1 time: 0.155933 Sep 04 01:15:23.255834 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 581/1188] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:23 2022] GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf => generated 1451 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:15:23.277581 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-3c40bc0c-3174-46ce-a3e8-bec671541872 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:23.278338 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-3c40bc0c-3174-46ce-a3e8-bec671541872 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 134e1ff6-ab5a-41a8-9afb-8891869494bf] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:23.285670 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3c40bc0c-3174-46ce-a3e8-bec671541872 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:23.286154 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3c40bc0c-3174-46ce-a3e8-bec671541872 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:23.292084 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3c40bc0c-3174-46ce-a3e8-bec671541872 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:23.292084 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3c40bc0c-3174-46ce-a3e8-bec671541872 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:23.292084 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3c40bc0c-3174-46ce-a3e8-bec671541872 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:23.292084 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-3c40bc0c-3174-46ce-a3e8-bec671541872 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:23.419813 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-3c40bc0c-3174-46ce-a3e8-bec671541872 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:23.428369 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-3c40bc0c-3174-46ce-a3e8-bec671541872 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf" status: 200 len: 1451 microversion: 2.1 time: 0.154871 Sep 04 01:15:23.428941 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 608/1189] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:23 2022] GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf => generated 1451 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:15:26.067385 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-29213796-d0b5-4567-8eaf-cb0e5e037026 req-7a62fb54-c9af-4c41-8180-8a1a5f7cf9af service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "134e1ff6-ab5a-41a8-9afb-8891869494bf", "tag": "8ad76ee4-45f9-4f07-93dd-75ebf95bbe32"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:15:26.077635 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-29213796-d0b5-4567-8eaf-cb0e5e037026 req-7a62fb54-c9af-4c41-8180-8a1a5f7cf9af service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:26.078114 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-29213796-d0b5-4567-8eaf-cb0e5e037026 req-7a62fb54-c9af-4c41-8180-8a1a5f7cf9af service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:26.078548 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-29213796-d0b5-4567-8eaf-cb0e5e037026 req-7a62fb54-c9af-4c41-8180-8a1a5f7cf9af service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:26.090036 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-29213796-d0b5-4567-8eaf-cb0e5e037026 req-7a62fb54-c9af-4c41-8180-8a1a5f7cf9af service nova] Creating event network-changed:8ad76ee4-45f9-4f07-93dd-75ebf95bbe32 for instance 134e1ff6-ab5a-41a8-9afb-8891869494bf on hv-cinder-maste Sep 04 01:15:26.095414 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-29213796-d0b5-4567-8eaf-cb0e5e037026 req-7a62fb54-c9af-4c41-8180-8a1a5f7cf9af service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032610 Sep 04 01:15:26.096204 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 582/1190] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:15:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:15:39.060623 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-71723c9b-80d3-4714-9902-00c732ee52d2 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:15:39.062916 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-71723c9b-80d3-4714-9902-00c732ee52d2 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:39.066094 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_api:SELECT=17 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:15:39.066784 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=43 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:15:39.072732 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-71723c9b-80d3-4714-9902-00c732ee52d2 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:39.077296 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-71723c9b-80d3-4714-9902-00c732ee52d2 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:39.077296 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-71723c9b-80d3-4714-9902-00c732ee52d2 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:39.077296 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-71723c9b-80d3-4714-9902-00c732ee52d2 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:39.077296 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-71723c9b-80d3-4714-9902-00c732ee52d2 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:39.077737 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-71723c9b-80d3-4714-9902-00c732ee52d2 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:39.298588 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-71723c9b-80d3-4714-9902-00c732ee52d2 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "POST /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64/action" status: 202 len: 0 microversion: 2.1 time: 0.241898 Sep 04 01:15:39.299274 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 609/1191] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Sep 4 01:15:39 2022] POST /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64/action => generated 0 bytes in 243 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 04 01:15:39.309406 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-0019ae8e-d762-4317-8158-dcc982502e90 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:39.312422 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-0019ae8e-d762-4317-8158-dcc982502e90 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:39.312943 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 04 01:15:39.313106 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Sun Sep 4 01:15:39 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 (ip 10.222.0.42) !!! Sep 04 01:15:39.314444 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_api:SELECT=21 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:15:39.315126 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=55 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:15:39.321587 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0019ae8e-d762-4317-8158-dcc982502e90 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:39.322057 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0019ae8e-d762-4317-8158-dcc982502e90 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:39.322485 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0019ae8e-d762-4317-8158-dcc982502e90 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:39.323172 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0019ae8e-d762-4317-8158-dcc982502e90 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:39.323617 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0019ae8e-d762-4317-8158-dcc982502e90 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:39.324042 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0019ae8e-d762-4317-8158-dcc982502e90 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:39.477634 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-0019ae8e-d762-4317-8158-dcc982502e90 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:39.487489 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-0019ae8e-d762-4317-8158-dcc982502e90 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1752 microversion: 2.1 time: 0.182862 Sep 04 01:15:39.488529 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 583/1192] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:39 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1752 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:15:40.515610 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-d5d1c3d0-2fe3-47a5-93e8-6ac685e2e20d tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:40.516304 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-d5d1c3d0-2fe3-47a5-93e8-6ac685e2e20d tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:40.522450 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d5d1c3d0-2fe3-47a5-93e8-6ac685e2e20d tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:40.522947 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d5d1c3d0-2fe3-47a5-93e8-6ac685e2e20d tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:40.523494 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d5d1c3d0-2fe3-47a5-93e8-6ac685e2e20d tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:40.524195 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d5d1c3d0-2fe3-47a5-93e8-6ac685e2e20d tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:40.524733 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d5d1c3d0-2fe3-47a5-93e8-6ac685e2e20d tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:40.525182 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d5d1c3d0-2fe3-47a5-93e8-6ac685e2e20d tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:40.664847 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-d5d1c3d0-2fe3-47a5-93e8-6ac685e2e20d tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:40.673116 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-d5d1c3d0-2fe3-47a5-93e8-6ac685e2e20d tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1757 microversion: 2.1 time: 0.162193 Sep 04 01:15:40.673686 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 610/1193] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:40 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1757 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:15:41.698328 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-b1092a78-66f0-46da-8925-12300a63c4da tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:41.698937 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-b1092a78-66f0-46da-8925-12300a63c4da tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:41.705631 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b1092a78-66f0-46da-8925-12300a63c4da tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:41.706107 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b1092a78-66f0-46da-8925-12300a63c4da tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:41.706540 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b1092a78-66f0-46da-8925-12300a63c4da tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:41.707228 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b1092a78-66f0-46da-8925-12300a63c4da tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:41.707675 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b1092a78-66f0-46da-8925-12300a63c4da tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:41.708098 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b1092a78-66f0-46da-8925-12300a63c4da tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:41.863276 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-b1092a78-66f0-46da-8925-12300a63c4da tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:41.872034 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-b1092a78-66f0-46da-8925-12300a63c4da tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1757 microversion: 2.1 time: 0.178360 Sep 04 01:15:41.872603 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 584/1194] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:41 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1757 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:15:42.894741 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-ffa97cb9-8ff0-41af-ad7b-65bd62970a00 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:42.895353 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-ffa97cb9-8ff0-41af-ad7b-65bd62970a00 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:42.901551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ffa97cb9-8ff0-41af-ad7b-65bd62970a00 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:42.902029 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ffa97cb9-8ff0-41af-ad7b-65bd62970a00 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:42.902464 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ffa97cb9-8ff0-41af-ad7b-65bd62970a00 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:42.903148 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ffa97cb9-8ff0-41af-ad7b-65bd62970a00 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:42.903600 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ffa97cb9-8ff0-41af-ad7b-65bd62970a00 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:42.904026 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ffa97cb9-8ff0-41af-ad7b-65bd62970a00 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:43.046607 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ffa97cb9-8ff0-41af-ad7b-65bd62970a00 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:43.055668 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-ffa97cb9-8ff0-41af-ad7b-65bd62970a00 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1757 microversion: 2.1 time: 0.164824 Sep 04 01:15:43.056258 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 611/1195] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:42 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1757 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:15:44.080950 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-63225762-d0ba-4190-9914-28438aea0409 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:44.081599 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-63225762-d0ba-4190-9914-28438aea0409 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:44.088656 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-63225762-d0ba-4190-9914-28438aea0409 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:44.089162 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-63225762-d0ba-4190-9914-28438aea0409 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:44.089594 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-63225762-d0ba-4190-9914-28438aea0409 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:44.090276 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-63225762-d0ba-4190-9914-28438aea0409 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:44.090723 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-63225762-d0ba-4190-9914-28438aea0409 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:44.091139 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-63225762-d0ba-4190-9914-28438aea0409 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:44.233279 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-63225762-d0ba-4190-9914-28438aea0409 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:44.241530 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-63225762-d0ba-4190-9914-28438aea0409 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1757 microversion: 2.1 time: 0.164430 Sep 04 01:15:44.242110 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 585/1196] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:44 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1757 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:15:45.265172 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-69364a96-1543-4ec6-94c2-5e153bc9066a tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:45.265782 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-69364a96-1543-4ec6-94c2-5e153bc9066a tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:45.272103 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-69364a96-1543-4ec6-94c2-5e153bc9066a tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:45.272581 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-69364a96-1543-4ec6-94c2-5e153bc9066a tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:45.273028 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-69364a96-1543-4ec6-94c2-5e153bc9066a tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:45.273712 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-69364a96-1543-4ec6-94c2-5e153bc9066a tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:45.274159 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-69364a96-1543-4ec6-94c2-5e153bc9066a tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:45.274573 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-69364a96-1543-4ec6-94c2-5e153bc9066a tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:45.414433 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-69364a96-1543-4ec6-94c2-5e153bc9066a tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:45.422659 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-69364a96-1543-4ec6-94c2-5e153bc9066a tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1757 microversion: 2.1 time: 0.161369 Sep 04 01:15:45.423227 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 612/1197] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:45 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1757 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:15:46.446295 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-348aefac-8e40-4833-bb0e-3b2cefc1f106 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:46.446899 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-348aefac-8e40-4833-bb0e-3b2cefc1f106 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:46.453332 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-348aefac-8e40-4833-bb0e-3b2cefc1f106 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:46.453808 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-348aefac-8e40-4833-bb0e-3b2cefc1f106 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:46.454239 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-348aefac-8e40-4833-bb0e-3b2cefc1f106 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:46.454923 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-348aefac-8e40-4833-bb0e-3b2cefc1f106 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:46.455370 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-348aefac-8e40-4833-bb0e-3b2cefc1f106 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:46.455789 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-348aefac-8e40-4833-bb0e-3b2cefc1f106 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:46.597497 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-348aefac-8e40-4833-bb0e-3b2cefc1f106 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:46.605912 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-348aefac-8e40-4833-bb0e-3b2cefc1f106 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1757 microversion: 2.1 time: 0.163504 Sep 04 01:15:46.606502 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 586/1198] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:46 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1757 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:15:47.629544 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-a3dba0d9-e0c1-4edf-8c2c-8041b5ded317 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:47.630147 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-a3dba0d9-e0c1-4edf-8c2c-8041b5ded317 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:47.636504 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a3dba0d9-e0c1-4edf-8c2c-8041b5ded317 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:47.637258 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a3dba0d9-e0c1-4edf-8c2c-8041b5ded317 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:47.637689 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a3dba0d9-e0c1-4edf-8c2c-8041b5ded317 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:47.638369 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a3dba0d9-e0c1-4edf-8c2c-8041b5ded317 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:47.638812 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a3dba0d9-e0c1-4edf-8c2c-8041b5ded317 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:47.639230 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-a3dba0d9-e0c1-4edf-8c2c-8041b5ded317 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:47.780199 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-a3dba0d9-e0c1-4edf-8c2c-8041b5ded317 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:47.788286 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-a3dba0d9-e0c1-4edf-8c2c-8041b5ded317 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1757 microversion: 2.1 time: 0.162587 Sep 04 01:15:47.789517 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 613/1199] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:47 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1757 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:15:48.813598 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-c339dd4e-509c-44fe-880a-b715a9e85309 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:48.813598 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-c339dd4e-509c-44fe-880a-b715a9e85309 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:48.819429 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c339dd4e-509c-44fe-880a-b715a9e85309 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:48.819901 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c339dd4e-509c-44fe-880a-b715a9e85309 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:48.820445 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c339dd4e-509c-44fe-880a-b715a9e85309 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:48.821143 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c339dd4e-509c-44fe-880a-b715a9e85309 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:48.821588 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c339dd4e-509c-44fe-880a-b715a9e85309 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:48.822004 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-c339dd4e-509c-44fe-880a-b715a9e85309 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:48.982026 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-c339dd4e-509c-44fe-880a-b715a9e85309 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:48.990349 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-c339dd4e-509c-44fe-880a-b715a9e85309 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1757 microversion: 2.1 time: 0.181738 Sep 04 01:15:48.990928 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 587/1200] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:48 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1757 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:15:50.014534 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-62c79fd5-3bfe-4789-8b9e-cfb4d66739a8 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:50.015126 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-62c79fd5-3bfe-4789-8b9e-cfb4d66739a8 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:50.021654 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-62c79fd5-3bfe-4789-8b9e-cfb4d66739a8 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:50.022125 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-62c79fd5-3bfe-4789-8b9e-cfb4d66739a8 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:50.022555 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-62c79fd5-3bfe-4789-8b9e-cfb4d66739a8 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:50.023772 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-62c79fd5-3bfe-4789-8b9e-cfb4d66739a8 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:50.024222 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-62c79fd5-3bfe-4789-8b9e-cfb4d66739a8 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:50.024634 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-62c79fd5-3bfe-4789-8b9e-cfb4d66739a8 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:50.169584 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-62c79fd5-3bfe-4789-8b9e-cfb4d66739a8 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:50.178091 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-62c79fd5-3bfe-4789-8b9e-cfb4d66739a8 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1757 microversion: 2.1 time: 0.167364 Sep 04 01:15:50.178670 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 614/1201] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:50 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1757 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:15:51.201877 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-7a164986-62d3-4017-918c-41e6f7175c30 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:51.202473 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-7a164986-62d3-4017-918c-41e6f7175c30 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:51.208964 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7a164986-62d3-4017-918c-41e6f7175c30 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:51.209473 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7a164986-62d3-4017-918c-41e6f7175c30 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:51.209911 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7a164986-62d3-4017-918c-41e6f7175c30 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:51.210602 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7a164986-62d3-4017-918c-41e6f7175c30 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:51.211371 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7a164986-62d3-4017-918c-41e6f7175c30 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:51.211803 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-7a164986-62d3-4017-918c-41e6f7175c30 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:51.352565 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-7a164986-62d3-4017-918c-41e6f7175c30 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:51.361488 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-7a164986-62d3-4017-918c-41e6f7175c30 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1757 microversion: 2.1 time: 0.163449 Sep 04 01:15:51.362386 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 588/1202] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:51 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1757 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:15:52.386839 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-51e1cafe-7b1a-4edc-813f-515ee3801fee tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:52.387429 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-51e1cafe-7b1a-4edc-813f-515ee3801fee tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:52.393946 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-51e1cafe-7b1a-4edc-813f-515ee3801fee tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:52.394426 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-51e1cafe-7b1a-4edc-813f-515ee3801fee tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:52.394866 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-51e1cafe-7b1a-4edc-813f-515ee3801fee tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:52.395548 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-51e1cafe-7b1a-4edc-813f-515ee3801fee tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:52.395993 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-51e1cafe-7b1a-4edc-813f-515ee3801fee tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:52.396413 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-51e1cafe-7b1a-4edc-813f-515ee3801fee tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:52.537510 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-51e1cafe-7b1a-4edc-813f-515ee3801fee tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:52.546526 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-51e1cafe-7b1a-4edc-813f-515ee3801fee tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1757 microversion: 2.1 time: 0.164131 Sep 04 01:15:52.547106 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 615/1203] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:52 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1757 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:15:53.571181 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-f491d919-d2f5-4e8e-86bc-02ccceccc01c tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:53.571789 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-f491d919-d2f5-4e8e-86bc-02ccceccc01c tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:53.579162 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f491d919-d2f5-4e8e-86bc-02ccceccc01c tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:53.579674 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f491d919-d2f5-4e8e-86bc-02ccceccc01c tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:53.580111 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f491d919-d2f5-4e8e-86bc-02ccceccc01c tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:53.580803 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f491d919-d2f5-4e8e-86bc-02ccceccc01c tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:53.581320 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f491d919-d2f5-4e8e-86bc-02ccceccc01c tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:53.581745 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f491d919-d2f5-4e8e-86bc-02ccceccc01c tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:53.725818 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-f491d919-d2f5-4e8e-86bc-02ccceccc01c tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:53.734298 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-f491d919-d2f5-4e8e-86bc-02ccceccc01c tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1757 microversion: 2.1 time: 0.167054 Sep 04 01:15:53.734881 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 589/1204] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:53 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1757 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:15:54.758573 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-47bbe89c-603e-4394-86de-5fc1b17cc603 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:54.759138 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-47bbe89c-603e-4394-86de-5fc1b17cc603 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:54.765743 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-47bbe89c-603e-4394-86de-5fc1b17cc603 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:54.766222 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-47bbe89c-603e-4394-86de-5fc1b17cc603 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:54.766655 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-47bbe89c-603e-4394-86de-5fc1b17cc603 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:54.767371 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-47bbe89c-603e-4394-86de-5fc1b17cc603 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:54.767821 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-47bbe89c-603e-4394-86de-5fc1b17cc603 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:54.768239 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-47bbe89c-603e-4394-86de-5fc1b17cc603 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:54.926929 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-47bbe89c-603e-4394-86de-5fc1b17cc603 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:54.936450 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-47bbe89c-603e-4394-86de-5fc1b17cc603 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1757 microversion: 2.1 time: 0.182046 Sep 04 01:15:54.937025 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 616/1205] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:54 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1757 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:15:55.959719 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-35d50b6a-9c80-44d3-9d99-8649d5a13d88 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:55.960364 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-35d50b6a-9c80-44d3-9d99-8649d5a13d88 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:55.967087 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-35d50b6a-9c80-44d3-9d99-8649d5a13d88 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:55.967568 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-35d50b6a-9c80-44d3-9d99-8649d5a13d88 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:55.968005 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-35d50b6a-9c80-44d3-9d99-8649d5a13d88 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:55.968696 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-35d50b6a-9c80-44d3-9d99-8649d5a13d88 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:55.969181 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-35d50b6a-9c80-44d3-9d99-8649d5a13d88 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:55.969611 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-35d50b6a-9c80-44d3-9d99-8649d5a13d88 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:56.110622 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-35d50b6a-9c80-44d3-9d99-8649d5a13d88 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:56.119378 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-35d50b6a-9c80-44d3-9d99-8649d5a13d88 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1757 microversion: 2.1 time: 0.163560 Sep 04 01:15:56.119955 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 590/1206] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:55 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1757 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:15:57.143508 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-e42e1d43-e354-4760-b3cf-f4895b1b0797 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:15:57.144134 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-e42e1d43-e354-4760-b3cf-f4895b1b0797 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:15:57.150684 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e42e1d43-e354-4760-b3cf-f4895b1b0797 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:57.151156 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e42e1d43-e354-4760-b3cf-f4895b1b0797 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:57.151586 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e42e1d43-e354-4760-b3cf-f4895b1b0797 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:57.152269 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e42e1d43-e354-4760-b3cf-f4895b1b0797 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:15:57.152713 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e42e1d43-e354-4760-b3cf-f4895b1b0797 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:15:57.153162 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-e42e1d43-e354-4760-b3cf-f4895b1b0797 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:15:57.303742 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-e42e1d43-e354-4760-b3cf-f4895b1b0797 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:15:57.312569 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-e42e1d43-e354-4760-b3cf-f4895b1b0797 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1760 microversion: 2.1 time: 0.172964 Sep 04 01:15:57.313148 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 617/1207] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:15:57 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1760 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:16:29.958059 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-924b01d9-ecea-4fc6-9db9-1db7d4fc41e0 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-264434114"}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:16:29.958699 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-924b01d9-ecea-4fc6-9db9-1db7d4fc41e0 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:16:29.960832 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_api:SELECT=16 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:16:29.961627 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=48 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:16:29.966757 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-924b01d9-ecea-4fc6-9db9-1db7d4fc41e0 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:16:29.967230 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-924b01d9-ecea-4fc6-9db9-1db7d4fc41e0 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:16:29.967659 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-924b01d9-ecea-4fc6-9db9-1db7d4fc41e0 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:16:29.968334 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-924b01d9-ecea-4fc6-9db9-1db7d4fc41e0 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:16:29.968774 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-924b01d9-ecea-4fc6-9db9-1db7d4fc41e0 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:16:29.969222 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-924b01d9-ecea-4fc6-9db9-1db7d4fc41e0 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:16:30.096745 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.network.security_group_api [None req-924b01d9-ecea-4fc6-9db9-1db7d4fc41e0 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Removing security group 453a92ae-fa58-40a9-8940-f7ae3987450a from port 17937f22-477f-47ec-b6b4-0ea96fee6512 Sep 04 01:16:30.449464 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-924b01d9-ecea-4fc6-9db9-1db7d4fc41e0 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "POST /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64/action" status: 202 len: 0 microversion: 2.1 time: 0.495445 Sep 04 01:16:30.450089 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 591/1208] 10.222.0.42 () {60 vars in 1278 bytes} [Sun Sep 4 01:16:29 2022] POST /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64/action => generated 0 bytes in 497 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 04 01:16:30.461462 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_api:SELECT=18 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:16:30.463002 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=58,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:16:30.630251 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-f1254ff2-4e18-45fc-805d-4aade0ff80ff req-b05b7f20-17eb-4510-a6c4-f9b779eb702d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64", "tag": "17937f22-477f-47ec-b6b4-0ea96fee6512"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:16:30.641464 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-f1254ff2-4e18-45fc-805d-4aade0ff80ff req-b05b7f20-17eb-4510-a6c4-f9b779eb702d service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:16:30.642314 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-f1254ff2-4e18-45fc-805d-4aade0ff80ff req-b05b7f20-17eb-4510-a6c4-f9b779eb702d service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:16:30.642744 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-f1254ff2-4e18-45fc-805d-4aade0ff80ff req-b05b7f20-17eb-4510-a6c4-f9b779eb702d service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:16:30.654002 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-f1254ff2-4e18-45fc-805d-4aade0ff80ff req-b05b7f20-17eb-4510-a6c4-f9b779eb702d service nova] Creating event network-changed:17937f22-477f-47ec-b6b4-0ea96fee6512 for instance 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 on hv-cinder-maste Sep 04 01:16:30.659296 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-f1254ff2-4e18-45fc-805d-4aade0ff80ff req-b05b7f20-17eb-4510-a6c4-f9b779eb702d service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.203110 Sep 04 01:16:30.660377 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 618/1209] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:16:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:16:32.195987 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-8830c1f4-276d-49a6-87c6-cb40fca761ab tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:16:32.197126 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-8830c1f4-276d-49a6-87c6-cb40fca761ab tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:16:32.203260 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8830c1f4-276d-49a6-87c6-cb40fca761ab tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:16:32.203733 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8830c1f4-276d-49a6-87c6-cb40fca761ab tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:16:32.204161 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8830c1f4-276d-49a6-87c6-cb40fca761ab tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:16:32.204844 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8830c1f4-276d-49a6-87c6-cb40fca761ab tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:16:32.205313 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8830c1f4-276d-49a6-87c6-cb40fca761ab tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:16:32.205762 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8830c1f4-276d-49a6-87c6-cb40fca761ab tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:16:32.287182 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.objects.instance [None req-8830c1f4-276d-49a6-87c6-cb40fca761ab tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lazy-loading 'services' on Instance uuid 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 {{(pid=121037) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:16:32.453901 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-8830c1f4-276d-49a6-87c6-cb40fca761ab tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "DELETE /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64/os-volume_attachments/c0024b07-c22a-4a55-95ca-a833025840e2" status: 202 len: 0 microversion: 2.1 time: 0.262161 Sep 04 01:16:32.454547 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 592/1210] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Sep 4 01:16:32 2022] DELETE /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64/os-volume_attachments/c0024b07-c22a-4a55-95ca-a833025840e2 => generated 0 bytes in 264 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 04 01:16:32.467486 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-776134b3-4084-4518-b520-a8d20a309892 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:16:32.470306 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-776134b3-4084-4518-b520-a8d20a309892 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:16:32.480143 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-776134b3-4084-4518-b520-a8d20a309892 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:16:32.483153 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-776134b3-4084-4518-b520-a8d20a309892 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:16:32.483153 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-776134b3-4084-4518-b520-a8d20a309892 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:16:32.483153 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-776134b3-4084-4518-b520-a8d20a309892 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:16:32.483153 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-776134b3-4084-4518-b520-a8d20a309892 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:16:32.483550 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-776134b3-4084-4518-b520-a8d20a309892 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:16:32.513674 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-776134b3-4084-4518-b520-a8d20a309892 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054232 Sep 04 01:16:32.514325 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 619/1211] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 4 01:16:32 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:16:32.674090 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-952c2802-2413-4a11-854f-de17f8679685 req-a63500d5-3530-49e3-aeaa-625bcca4b368 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64", "tag": "17937f22-477f-47ec-b6b4-0ea96fee6512"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:16:32.683948 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-952c2802-2413-4a11-854f-de17f8679685 req-a63500d5-3530-49e3-aeaa-625bcca4b368 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:16:32.684423 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-952c2802-2413-4a11-854f-de17f8679685 req-a63500d5-3530-49e3-aeaa-625bcca4b368 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:16:32.684853 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-952c2802-2413-4a11-854f-de17f8679685 req-a63500d5-3530-49e3-aeaa-625bcca4b368 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:16:32.695652 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-952c2802-2413-4a11-854f-de17f8679685 req-a63500d5-3530-49e3-aeaa-625bcca4b368 service nova] Creating event network-changed:17937f22-477f-47ec-b6b4-0ea96fee6512 for instance 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 on hv-cinder-maste Sep 04 01:16:32.701261 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-952c2802-2413-4a11-854f-de17f8679685 req-a63500d5-3530-49e3-aeaa-625bcca4b368 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031161 Sep 04 01:16:32.701868 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 593/1212] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:16:32 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:16:33.527824 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-257462b6-123d-49b6-9b17-667e600a8f09 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:16:33.531133 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-257462b6-123d-49b6-9b17-667e600a8f09 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:16:33.537816 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-257462b6-123d-49b6-9b17-667e600a8f09 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:16:33.538300 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-257462b6-123d-49b6-9b17-667e600a8f09 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:16:33.538734 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-257462b6-123d-49b6-9b17-667e600a8f09 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:16:33.539422 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-257462b6-123d-49b6-9b17-667e600a8f09 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:16:33.539867 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-257462b6-123d-49b6-9b17-667e600a8f09 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:16:33.540308 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-257462b6-123d-49b6-9b17-667e600a8f09 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:16:33.573453 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-257462b6-123d-49b6-9b17-667e600a8f09 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050303 Sep 04 01:16:33.574004 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 620/1213] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 4 01:16:33 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:16:34.587910 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-5b5147b3-d3f4-4d27-8e10-cc62ada8c150 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:16:34.590249 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-5b5147b3-d3f4-4d27-8e10-cc62ada8c150 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:16:34.596346 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5b5147b3-d3f4-4d27-8e10-cc62ada8c150 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:16:34.596821 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5b5147b3-d3f4-4d27-8e10-cc62ada8c150 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:16:34.597283 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5b5147b3-d3f4-4d27-8e10-cc62ada8c150 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:16:34.597967 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5b5147b3-d3f4-4d27-8e10-cc62ada8c150 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:16:34.598413 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5b5147b3-d3f4-4d27-8e10-cc62ada8c150 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:16:34.598833 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-5b5147b3-d3f4-4d27-8e10-cc62ada8c150 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:16:34.627866 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-5b5147b3-d3f4-4d27-8e10-cc62ada8c150 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.045170 Sep 04 01:16:34.628416 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 594/1214] 10.222.0.42 () {58 vars in 1302 bytes} [Sun Sep 4 01:16:34 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64/os-volume_attachments => generated 25 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Sep 04 01:16:34.639163 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-2427d347-2bff-419c-9977-8a6b92c0eb19 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:16:34.639656 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-2427d347-2bff-419c-9977-8a6b92c0eb19 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:16:34.645984 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2427d347-2bff-419c-9977-8a6b92c0eb19 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:16:34.646453 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2427d347-2bff-419c-9977-8a6b92c0eb19 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:16:34.646880 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2427d347-2bff-419c-9977-8a6b92c0eb19 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:16:34.647556 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2427d347-2bff-419c-9977-8a6b92c0eb19 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:16:34.648000 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2427d347-2bff-419c-9977-8a6b92c0eb19 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:16:34.648418 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-2427d347-2bff-419c-9977-8a6b92c0eb19 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:16:34.722377 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.wsgi [None req-2427d347-2bff-419c-9977-8a6b92c0eb19 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] HTTP exception thrown: Instance 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 is not attached to volume c0024b07-c22a-4a55-95ca-a833025840e2 Sep 04 01:16:34.724693 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-2427d347-2bff-419c-9977-8a6b92c0eb19 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Request method failure captured: Sep 04 01:16:34.724693 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: request: DELETE /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64/os-volume_attachments/c0024b07-c22a-4a55-95ca-a833025840e2 HTTP/1.1 Sep 04 01:16:34.724693 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept: application/json Sep 04 01:16:34.724693 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept-Encoding: identity Sep 04 01:16:34.724693 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Connection: close Sep 04 01:16:34.724693 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Length: 0 Sep 04 01:16:34.724693 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Type: application/json Sep 04 01:16:34.724693 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Host: 10.222.0.42 Sep 04 01:16:34.724693 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Openstack-System-Scope: None Sep 04 01:16:34.724693 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: User-Agent: python-urllib3/1.26.12 Sep 04 01:16:34.724693 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Auth-Token: gAAAAABjE_vnL0pV-1RW9Er-uDiR5e6RYaXSjqhj5Szffn-65RfhToyMQFsJePXBuGApzimhzkdhIKIy4mKwLsTNViUfcAoFcZ99R1Y0lO-_LYYbbC-KZCoarDwCa-K12LoxX5tiOLd26zSMAjiXT4nXy1cKPfhHvt8adBrlZTI4lK8UlqJgP_s Sep 04 01:16:34.724693 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Id: None Sep 04 01:16:34.724693 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Name: None Sep 04 01:16:34.724693 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Identity-Status: Confirmed Sep 04 01:16:34.724693 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Is-Admin-Project: True Sep 04 01:16:34.724693 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Id: default Sep 04 01:16:34.724693 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Name: Default Sep 04 01:16:34.724693 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Id: 027792a6118140099aaf522ca15bb07e Sep 04 01:16:34.724693 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Name: tempest-TestMinimumBasicScenario-391848128 Sep 04 01:16:34.724693 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Role: reader,member Sep 04 01:16:34.724693 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Roles: reader,member Sep 04 01:16:34.726536 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/027792a6118140099aaf522ca15bb07e"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_027792a6118140099aaf522ca15bb07e", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/027792a6118140099aaf522ca15bb07e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/027792a6118140099aaf522ca15bb07e"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:16:34.726536 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant: tempest-TestMinimumBasicScenario-391848128 Sep 04 01:16:34.726536 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Id: 027792a6118140099aaf522ca15bb07e Sep 04 01:16:34.726536 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Name: tempest-TestMinimumBasicScenario-391848128 Sep 04 01:16:34.726536 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User: tempest-TestMinimumBasicScenario-391848128-project Sep 04 01:16:34.726536 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Id: default Sep 04 01:16:34.726536 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Name: Default Sep 04 01:16:34.726536 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Id: 9b5efa37b94b47949344d00ff88be4fb Sep 04 01:16:34.726536 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Name: tempest-TestMinimumBasicScenario-391848128-project Sep 04 01:16:34.726536 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: method: > Sep 04 01:16:34.726536 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: exception: Instance 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 is not attached to volume c0024b07-c22a-4a55-95ca-a833025840e2 Sep 04 01:16:34.728186 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: action_args: {'server_id': '3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64', 'id': 'c0024b07-c22a-4a55-95ca-a833025840e2'} Sep 04 01:16:34.728186 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:16:34.728186 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:16:34.728186 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Sep 04 01:16:34.728186 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Sep 04 01:16:34.728186 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 04 01:16:34.728186 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 04 01:16:34.728186 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Sep 04 01:16:34.728186 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Sep 04 01:16:34.728186 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id c0024b07-c22a-4a55-95ca-a833025840e2. Sep 04 01:16:34.728186 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:16:34.728186 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:16:34.728186 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:16:34.728186 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:16:34.728186 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:16:34.728186 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:16:34.728186 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:16:34.728186 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:16:34.728186 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:16:34.729808 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:16:34.729808 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Sep 04 01:16:34.729808 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Sep 04 01:16:34.729808 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 is not attached to volume c0024b07-c22a-4a55-95ca-a833025840e2 Sep 04 01:16:34.729808 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:16:34.729808 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:16:34.729808 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:16:34.729808 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:16:34.729808 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:16:34.729808 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:16:34.729808 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:16:34.729808 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:16:34.729808 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 is not attached to volume c0024b07-c22a-4a55-95ca-a833025840e2 Sep 04 01:16:34.729808 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:16:34.729808 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-2427d347-2bff-419c-9977-8a6b92c0eb19 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Returning 404 to user: Instance 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 is not attached to volume c0024b07-c22a-4a55-95ca-a833025840e2 {{(pid=121036) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:16:34.731465 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-2427d347-2bff-419c-9977-8a6b92c0eb19 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "DELETE /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64/os-volume_attachments/c0024b07-c22a-4a55-95ca-a833025840e2" status: 404 len: 154 microversion: 2.1 time: 0.090904 Sep 04 01:16:34.731465 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 621/1215] 10.222.0.42 () {58 vars in 1416 bytes} [Sun Sep 4 01:16:34 2022] DELETE /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64/os-volume_attachments/c0024b07-c22a-4a55-95ca-a833025840e2 => generated 154 bytes in 92 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 04 01:16:35.989480 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-ee65fe69-76b3-4c76-9327-58c8c4dc2d65 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:16:35.991881 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-ee65fe69-76b3-4c76-9327-58c8c4dc2d65 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:16:35.997326 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ee65fe69-76b3-4c76-9327-58c8c4dc2d65 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:16:35.997752 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ee65fe69-76b3-4c76-9327-58c8c4dc2d65 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:16:35.998190 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ee65fe69-76b3-4c76-9327-58c8c4dc2d65 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:16:35.998883 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ee65fe69-76b3-4c76-9327-58c8c4dc2d65 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:16:35.999360 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ee65fe69-76b3-4c76-9327-58c8c4dc2d65 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:16:35.999784 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ee65fe69-76b3-4c76-9327-58c8c4dc2d65 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:16:36.025991 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-ee65fe69-76b3-4c76-9327-58c8c4dc2d65 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Going to try to terminate instance {{(pid=121037) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 04 01:16:36.357486 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-ee65fe69-76b3-4c76-9327-58c8c4dc2d65 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "DELETE /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 204 len: 0 microversion: 2.1 time: 0.371588 Sep 04 01:16:36.358098 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 595/1216] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 4 01:16:35 2022] DELETE /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 04 01:16:36.366518 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-f1d3e1ef-f1ba-4b29-8f10-d82c1a92b066 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:16:36.367165 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-f1d3e1ef-f1ba-4b29-8f10-d82c1a92b066 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:16:36.374106 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f1d3e1ef-f1ba-4b29-8f10-d82c1a92b066 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:16:36.374585 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f1d3e1ef-f1ba-4b29-8f10-d82c1a92b066 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:16:36.375042 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f1d3e1ef-f1ba-4b29-8f10-d82c1a92b066 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:16:36.375732 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f1d3e1ef-f1ba-4b29-8f10-d82c1a92b066 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:16:36.376183 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f1d3e1ef-f1ba-4b29-8f10-d82c1a92b066 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:16:36.376605 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-f1d3e1ef-f1ba-4b29-8f10-d82c1a92b066 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:16:36.506923 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-f1d3e1ef-f1ba-4b29-8f10-d82c1a92b066 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:16:36.515096 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-f1d3e1ef-f1ba-4b29-8f10-d82c1a92b066 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1555 microversion: 2.1 time: 0.152615 Sep 04 01:16:36.515651 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 622/1217] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:16:36 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1555 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:16:37.539648 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-596a282c-70b4-4b93-8fc5-53e8e3647ddc tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:16:37.641607 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-596a282c-70b4-4b93-8fc5-53e8e3647ddc tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:16:37.641607 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-596a282c-70b4-4b93-8fc5-53e8e3647ddc tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:16:37.641607 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-596a282c-70b4-4b93-8fc5-53e8e3647ddc tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:16:37.641607 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-596a282c-70b4-4b93-8fc5-53e8e3647ddc tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:16:37.642748 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-596a282c-70b4-4b93-8fc5-53e8e3647ddc tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:16:37.642748 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-596a282c-70b4-4b93-8fc5-53e8e3647ddc tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:16:37.642748 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-596a282c-70b4-4b93-8fc5-53e8e3647ddc tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:16:37.686753 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-596a282c-70b4-4b93-8fc5-53e8e3647ddc tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:16:37.694450 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-596a282c-70b4-4b93-8fc5-53e8e3647ddc tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1555 microversion: 2.1 time: 0.159124 Sep 04 01:16:37.695014 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 596/1218] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:16:37 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1555 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:16:38.688581 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-39dcf0a7-5697-4cd9-ace3-0b26e5632232 req-ad8ba4ae-5c1b-4e99-9d21-e1918d555f3c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64", "name": "network-vif-deleted", "tag": "17937f22-477f-47ec-b6b4-0ea96fee6512"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:16:38.698845 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-39dcf0a7-5697-4cd9-ace3-0b26e5632232 req-ad8ba4ae-5c1b-4e99-9d21-e1918d555f3c service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:16:38.699821 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-39dcf0a7-5697-4cd9-ace3-0b26e5632232 req-ad8ba4ae-5c1b-4e99-9d21-e1918d555f3c service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:16:38.699821 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-39dcf0a7-5697-4cd9-ace3-0b26e5632232 req-ad8ba4ae-5c1b-4e99-9d21-e1918d555f3c service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:16:38.710489 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-39dcf0a7-5697-4cd9-ace3-0b26e5632232 req-ad8ba4ae-5c1b-4e99-9d21-e1918d555f3c service nova] Creating event network-vif-deleted:17937f22-477f-47ec-b6b4-0ea96fee6512 for instance 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 on hv-cinder-maste Sep 04 01:16:38.716279 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-39dcf0a7-5697-4cd9-ace3-0b26e5632232 req-ad8ba4ae-5c1b-4e99-9d21-e1918d555f3c service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031475 Sep 04 01:16:38.716853 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 623/1219] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:16:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:16:38.719696 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-b5326649-9da6-4c7b-b548-e39b33f5d860 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:16:38.720238 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-b5326649-9da6-4c7b-b548-e39b33f5d860 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:16:38.726765 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b5326649-9da6-4c7b-b548-e39b33f5d860 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:16:38.728537 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b5326649-9da6-4c7b-b548-e39b33f5d860 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:16:38.728537 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b5326649-9da6-4c7b-b548-e39b33f5d860 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:16:38.728537 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b5326649-9da6-4c7b-b548-e39b33f5d860 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:16:38.728861 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b5326649-9da6-4c7b-b548-e39b33f5d860 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:16:38.729308 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b5326649-9da6-4c7b-b548-e39b33f5d860 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:16:38.792770 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-b5326649-9da6-4c7b-b548-e39b33f5d860 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9b5efa37b94b47949344d00ff88be4fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027792a6118140099aaf522ca15bb07e', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:16:38.800578 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-b5326649-9da6-4c7b-b548-e39b33f5d860 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 200 len: 1513 microversion: 2.1 time: 0.085477 Sep 04 01:16:38.801161 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 597/1220] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:16:38 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 1513 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:16:39.826766 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-d0281973-d951-4c55-a7e5-2b52c35c0c89 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:16:39.827528 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-d0281973-d951-4c55-a7e5-2b52c35c0c89 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] [instance: 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:16:39.833803 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d0281973-d951-4c55-a7e5-2b52c35c0c89 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:16:39.834280 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d0281973-d951-4c55-a7e5-2b52c35c0c89 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:16:39.834712 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d0281973-d951-4c55-a7e5-2b52c35c0c89 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:16:39.835397 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d0281973-d951-4c55-a7e5-2b52c35c0c89 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:16:39.835844 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d0281973-d951-4c55-a7e5-2b52c35c0c89 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:16:39.836273 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d0281973-d951-4c55-a7e5-2b52c35c0c89 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:16:39.849906 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.wsgi [None req-d0281973-d951-4c55-a7e5-2b52c35c0c89 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] HTTP exception thrown: Instance 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 could not be found. Sep 04 01:16:39.851520 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-d0281973-d951-4c55-a7e5-2b52c35c0c89 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Request method failure captured: Sep 04 01:16:39.851520 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: request: GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 HTTP/1.1 Sep 04 01:16:39.851520 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept: application/json Sep 04 01:16:39.851520 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept-Encoding: identity Sep 04 01:16:39.851520 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Connection: close Sep 04 01:16:39.851520 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Length: 0 Sep 04 01:16:39.851520 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Type: application/json Sep 04 01:16:39.851520 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Host: 10.222.0.42 Sep 04 01:16:39.851520 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Openstack-System-Scope: None Sep 04 01:16:39.851520 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: User-Agent: python-urllib3/1.26.12 Sep 04 01:16:39.851520 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Auth-Token: gAAAAABjE_vnL0pV-1RW9Er-uDiR5e6RYaXSjqhj5Szffn-65RfhToyMQFsJePXBuGApzimhzkdhIKIy4mKwLsTNViUfcAoFcZ99R1Y0lO-_LYYbbC-KZCoarDwCa-K12LoxX5tiOLd26zSMAjiXT4nXy1cKPfhHvt8adBrlZTI4lK8UlqJgP_s Sep 04 01:16:39.851520 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Id: None Sep 04 01:16:39.851520 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Name: None Sep 04 01:16:39.851520 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Identity-Status: Confirmed Sep 04 01:16:39.851520 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Is-Admin-Project: True Sep 04 01:16:39.851520 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Id: default Sep 04 01:16:39.851520 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Name: Default Sep 04 01:16:39.851520 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Id: 027792a6118140099aaf522ca15bb07e Sep 04 01:16:39.851520 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Name: tempest-TestMinimumBasicScenario-391848128 Sep 04 01:16:39.851520 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Role: reader,member Sep 04 01:16:39.851520 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Roles: reader,member Sep 04 01:16:39.853942 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/027792a6118140099aaf522ca15bb07e"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_027792a6118140099aaf522ca15bb07e", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/027792a6118140099aaf522ca15bb07e"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/027792a6118140099aaf522ca15bb07e"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:16:39.853942 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant: tempest-TestMinimumBasicScenario-391848128 Sep 04 01:16:39.853942 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Id: 027792a6118140099aaf522ca15bb07e Sep 04 01:16:39.853942 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Name: tempest-TestMinimumBasicScenario-391848128 Sep 04 01:16:39.853942 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User: tempest-TestMinimumBasicScenario-391848128-project Sep 04 01:16:39.853942 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Id: default Sep 04 01:16:39.853942 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Name: Default Sep 04 01:16:39.853942 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Id: 9b5efa37b94b47949344d00ff88be4fb Sep 04 01:16:39.853942 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Name: tempest-TestMinimumBasicScenario-391848128-project Sep 04 01:16:39.853942 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: method: > Sep 04 01:16:39.853942 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: exception: Instance 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 could not be found. Sep 04 01:16:39.853942 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: action_args: {'id': '3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64'} Sep 04 01:16:39.855043 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:16:39.855043 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:16:39.855043 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:16:39.855043 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:16:39.855043 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:16:39.855043 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:16:39.855043 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:16:39.855043 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:16:39.855043 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:16:39.855043 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:16:39.855043 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 could not be found. Sep 04 01:16:39.855043 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:16:39.855043 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:16:39.855043 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:16:39.855043 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:16:39.855043 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:16:39.855043 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:16:39.855043 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:16:39.855043 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:16:39.855043 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:16:39.856716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:16:39.856716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 04 01:16:39.856716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 04 01:16:39.856716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:16:39.856716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:16:39.856716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:16:39.856716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:16:39.856716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 could not be found. Sep 04 01:16:39.856716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:16:39.856716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:16:39.856716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:16:39.856716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:16:39.856716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:16:39.856716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:16:39.856716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:16:39.856716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:16:39.856716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 could not be found. Sep 04 01:16:39.856716 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:16:39.858458 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-d0281973-d951-4c55-a7e5-2b52c35c0c89 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Returning 404 to user: Instance 3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 could not be found. {{(pid=121036) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:16:39.858458 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-d0281973-d951-4c55-a7e5-2b52c35c0c89 tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64" status: 404 len: 111 microversion: 2.1 time: 0.030874 Sep 04 01:16:39.858458 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 624/1221] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:16:39 2022] GET /compute/v2.1/servers/3b8b93cc-1d4b-4d17-b4d2-e89f42b3be64 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 04 01:16:39.862317 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-4d8bd951-a781-484d-a0e2-911527dfd56c tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] Calling method '.version_select at 0x7fc10c6da0d0>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:16:39.967825 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-4d8bd951-a781-484d-a0e2-911527dfd56c tempest-TestMinimumBasicScenario-391848128 tempest-TestMinimumBasicScenario-391848128-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1802603257" status: 202 len: 0 microversion: 2.1 time: 0.109289 Sep 04 01:16:39.968410 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 598/1222] 10.222.0.42 () {58 vars in 1272 bytes} [Sun Sep 4 01:16:39 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-1802603257 => generated 0 bytes in 111 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 04 01:17:21.052992 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-1b8d1ca3-fa71-4cdd-8e86-9fe93c425f0e tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:17:21.054133 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-1b8d1ca3-fa71-4cdd-8e86-9fe93c425f0e tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 134e1ff6-ab5a-41a8-9afb-8891869494bf] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:17:21.056278 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_api:SELECT=14 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:17:21.057056 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=30 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:17:21.058056 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 04 01:17:21.058210 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Sun Sep 4 01:17:21 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf (ip 10.222.0.42) !!! Sep 04 01:17:21.071009 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1b8d1ca3-fa71-4cdd-8e86-9fe93c425f0e tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:21.071495 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1b8d1ca3-fa71-4cdd-8e86-9fe93c425f0e tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:21.071926 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1b8d1ca3-fa71-4cdd-8e86-9fe93c425f0e tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:21.072624 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1b8d1ca3-fa71-4cdd-8e86-9fe93c425f0e tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:21.073125 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1b8d1ca3-fa71-4cdd-8e86-9fe93c425f0e tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:21.073538 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-1b8d1ca3-fa71-4cdd-8e86-9fe93c425f0e tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:21.099220 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-1b8d1ca3-fa71-4cdd-8e86-9fe93c425f0e tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 134e1ff6-ab5a-41a8-9afb-8891869494bf] Going to try to terminate instance {{(pid=121036) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 04 01:17:21.411791 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-1b8d1ca3-fa71-4cdd-8e86-9fe93c425f0e tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "DELETE /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf" status: 204 len: 0 microversion: 2.1 time: 0.362743 Sep 04 01:17:21.412622 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 625/1223] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 4 01:17:21 2022] DELETE /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf => generated 0 bytes in 364 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 04 01:17:21.420612 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-2570df50-9a29-4815-9f3c-623fc779c53d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:17:21.421288 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-2570df50-9a29-4815-9f3c-623fc779c53d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 134e1ff6-ab5a-41a8-9afb-8891869494bf] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:17:21.423777 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=47,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:17:21.424652 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_api:SELECT=22,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:17:21.430173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2570df50-9a29-4815-9f3c-623fc779c53d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:21.430646 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2570df50-9a29-4815-9f3c-623fc779c53d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:21.431069 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2570df50-9a29-4815-9f3c-623fc779c53d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:21.431833 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2570df50-9a29-4815-9f3c-623fc779c53d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:21.432273 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2570df50-9a29-4815-9f3c-623fc779c53d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:21.432684 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2570df50-9a29-4815-9f3c-623fc779c53d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:21.575746 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-2570df50-9a29-4815-9f3c-623fc779c53d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:17:21.584898 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-2570df50-9a29-4815-9f3c-623fc779c53d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf" status: 200 len: 1576 microversion: 2.1 time: 0.168277 Sep 04 01:17:21.586530 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 599/1224] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:17:21 2022] GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf => generated 1576 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:17:22.610031 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-862b755b-2672-435a-bd9a-0e0992e7f9d5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:17:22.610626 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-862b755b-2672-435a-bd9a-0e0992e7f9d5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 134e1ff6-ab5a-41a8-9afb-8891869494bf] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:17:22.616971 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-862b755b-2672-435a-bd9a-0e0992e7f9d5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:22.617482 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-862b755b-2672-435a-bd9a-0e0992e7f9d5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:22.617920 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-862b755b-2672-435a-bd9a-0e0992e7f9d5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:22.618632 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-862b755b-2672-435a-bd9a-0e0992e7f9d5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:22.619080 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-862b755b-2672-435a-bd9a-0e0992e7f9d5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:22.619496 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-862b755b-2672-435a-bd9a-0e0992e7f9d5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:22.756294 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-862b755b-2672-435a-bd9a-0e0992e7f9d5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:17:22.764108 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-862b755b-2672-435a-bd9a-0e0992e7f9d5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf" status: 200 len: 1576 microversion: 2.1 time: 0.158995 Sep 04 01:17:22.764668 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 626/1225] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:17:22 2022] GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf => generated 1576 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:17:23.790697 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-b6a4cbf1-2c8d-4365-bb49-67a305ef5de2 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:17:23.791339 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-b6a4cbf1-2c8d-4365-bb49-67a305ef5de2 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 134e1ff6-ab5a-41a8-9afb-8891869494bf] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:17:23.810917 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b6a4cbf1-2c8d-4365-bb49-67a305ef5de2 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:23.810917 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b6a4cbf1-2c8d-4365-bb49-67a305ef5de2 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:23.810917 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b6a4cbf1-2c8d-4365-bb49-67a305ef5de2 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:23.810917 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b6a4cbf1-2c8d-4365-bb49-67a305ef5de2 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:23.812931 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b6a4cbf1-2c8d-4365-bb49-67a305ef5de2 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:23.812931 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b6a4cbf1-2c8d-4365-bb49-67a305ef5de2 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:23.875813 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-b6a4cbf1-2c8d-4365-bb49-67a305ef5de2 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:17:23.883503 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-b6a4cbf1-2c8d-4365-bb49-67a305ef5de2 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf" status: 200 len: 1508 microversion: 2.1 time: 0.098493 Sep 04 01:17:23.884069 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 600/1226] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:17:23 2022] GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf => generated 1508 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:17:24.085877 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-7cbdfaab-098c-44df-876f-fd84d8187491 req-8e28036b-1535-4935-bd44-0e0aa7728c6e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "134e1ff6-ab5a-41a8-9afb-8891869494bf", "name": "network-vif-deleted", "tag": "8ad76ee4-45f9-4f07-93dd-75ebf95bbe32"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:17:24.096434 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-7cbdfaab-098c-44df-876f-fd84d8187491 req-8e28036b-1535-4935-bd44-0e0aa7728c6e service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:24.096978 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-7cbdfaab-098c-44df-876f-fd84d8187491 req-8e28036b-1535-4935-bd44-0e0aa7728c6e service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:24.097385 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-7cbdfaab-098c-44df-876f-fd84d8187491 req-8e28036b-1535-4935-bd44-0e0aa7728c6e service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:24.108500 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-7cbdfaab-098c-44df-876f-fd84d8187491 req-8e28036b-1535-4935-bd44-0e0aa7728c6e service nova] Creating event network-vif-deleted:8ad76ee4-45f9-4f07-93dd-75ebf95bbe32 for instance 134e1ff6-ab5a-41a8-9afb-8891869494bf on hv-cinder-maste Sep 04 01:17:24.114569 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-7cbdfaab-098c-44df-876f-fd84d8187491 req-8e28036b-1535-4935-bd44-0e0aa7728c6e service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032653 Sep 04 01:17:24.115163 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 627/1227] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:17:24 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:17:24.907559 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-fbe9b0ec-68f9-4d07-b582-6385c7accade tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:17:24.908284 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-fbe9b0ec-68f9-4d07-b582-6385c7accade tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 134e1ff6-ab5a-41a8-9afb-8891869494bf] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:17:24.914812 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fbe9b0ec-68f9-4d07-b582-6385c7accade tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:24.915303 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fbe9b0ec-68f9-4d07-b582-6385c7accade tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:24.915759 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fbe9b0ec-68f9-4d07-b582-6385c7accade tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:24.916744 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fbe9b0ec-68f9-4d07-b582-6385c7accade tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:24.917263 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fbe9b0ec-68f9-4d07-b582-6385c7accade tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:24.917696 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fbe9b0ec-68f9-4d07-b582-6385c7accade tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:24.958102 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fbe9b0ec-68f9-4d07-b582-6385c7accade tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:24.958588 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fbe9b0ec-68f9-4d07-b582-6385c7accade tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:24.959019 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-fbe9b0ec-68f9-4d07-b582-6385c7accade tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:24.959463 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.objects.instance [None req-fbe9b0ec-68f9-4d07-b582-6385c7accade tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lazy-loading 'fault' on Instance uuid 134e1ff6-ab5a-41a8-9afb-8891869494bf {{(pid=121037) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:17:25.010687 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-fbe9b0ec-68f9-4d07-b582-6385c7accade tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:17:25.019383 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-fbe9b0ec-68f9-4d07-b582-6385c7accade tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf" status: 200 len: 1229 microversion: 2.1 time: 0.115781 Sep 04 01:17:25.019955 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 601/1228] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:17:24 2022] GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf => generated 1229 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:17:26.044359 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-9f42e361-9208-4e65-a9e6-345bbd1a8d20 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:17:26.045054 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-9f42e361-9208-4e65-a9e6-345bbd1a8d20 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 134e1ff6-ab5a-41a8-9afb-8891869494bf] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:17:26.052134 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9f42e361-9208-4e65-a9e6-345bbd1a8d20 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:26.052613 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9f42e361-9208-4e65-a9e6-345bbd1a8d20 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:26.053049 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9f42e361-9208-4e65-a9e6-345bbd1a8d20 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:26.053786 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9f42e361-9208-4e65-a9e6-345bbd1a8d20 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:26.054239 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9f42e361-9208-4e65-a9e6-345bbd1a8d20 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:26.054664 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9f42e361-9208-4e65-a9e6-345bbd1a8d20 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:26.067580 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.wsgi [None req-9f42e361-9208-4e65-a9e6-345bbd1a8d20 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] HTTP exception thrown: Instance 134e1ff6-ab5a-41a8-9afb-8891869494bf could not be found. Sep 04 01:17:26.069247 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-9f42e361-9208-4e65-a9e6-345bbd1a8d20 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Request method failure captured: Sep 04 01:17:26.069247 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: request: GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf HTTP/1.1 Sep 04 01:17:26.069247 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept: application/json Sep 04 01:17:26.069247 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept-Encoding: identity Sep 04 01:17:26.069247 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Connection: close Sep 04 01:17:26.069247 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Length: 0 Sep 04 01:17:26.069247 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Type: application/json Sep 04 01:17:26.069247 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Host: 10.222.0.42 Sep 04 01:17:26.069247 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Openstack-System-Scope: None Sep 04 01:17:26.069247 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: User-Agent: python-urllib3/1.26.12 Sep 04 01:17:26.069247 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Auth-Token: gAAAAABjE_v3xFr83GF5HAJz0F1ePckFApHoM5uGMmIbR4HzFsfIZD_EnhL2OGYcBnvrJRXBe6apIZ4muzHym6lqQRHT8CWNei8sBt1BevsxBM3QKX99qXYG3ttnjXaWqmmKrkT5XoKcl347t24GOiR5KkDxLGu96o2UlmXDRT33-ZG56RntQ2w Sep 04 01:17:26.069247 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Id: None Sep 04 01:17:26.069247 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Name: None Sep 04 01:17:26.069247 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Identity-Status: Confirmed Sep 04 01:17:26.069247 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Is-Admin-Project: True Sep 04 01:17:26.069247 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Id: default Sep 04 01:17:26.069247 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Name: Default Sep 04 01:17:26.069247 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Id: 2506b96ac13541e6a7a47c5aedca67e5 Sep 04 01:17:26.069247 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Name: tempest-TestVolumeBootPattern-1801805144 Sep 04 01:17:26.069247 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Role: reader,member Sep 04 01:17:26.069247 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Roles: reader,member Sep 04 01:17:26.071017 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2506b96ac13541e6a7a47c5aedca67e5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_2506b96ac13541e6a7a47c5aedca67e5", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2506b96ac13541e6a7a47c5aedca67e5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/2506b96ac13541e6a7a47c5aedca67e5"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:17:26.071017 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant: tempest-TestVolumeBootPattern-1801805144 Sep 04 01:17:26.071017 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Id: 2506b96ac13541e6a7a47c5aedca67e5 Sep 04 01:17:26.071017 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Name: tempest-TestVolumeBootPattern-1801805144 Sep 04 01:17:26.071017 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User: tempest-TestVolumeBootPattern-1801805144-project Sep 04 01:17:26.071017 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Id: default Sep 04 01:17:26.071017 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Name: Default Sep 04 01:17:26.071017 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Id: 48c4f0c78a364c00a793666eeb5a520c Sep 04 01:17:26.071017 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Name: tempest-TestVolumeBootPattern-1801805144-project Sep 04 01:17:26.071017 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: method: > Sep 04 01:17:26.071017 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: exception: Instance 134e1ff6-ab5a-41a8-9afb-8891869494bf could not be found. Sep 04 01:17:26.071017 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: action_args: {'id': '134e1ff6-ab5a-41a8-9afb-8891869494bf'} Sep 04 01:17:26.072187 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:17:26.072187 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:17:26.072187 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:17:26.072187 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:17:26.072187 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:17:26.072187 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:17:26.072187 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:17:26.072187 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:17:26.072187 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:17:26.072187 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:17:26.072187 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 134e1ff6-ab5a-41a8-9afb-8891869494bf could not be found. Sep 04 01:17:26.072187 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:17:26.072187 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:17:26.072187 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:17:26.072187 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:17:26.072187 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:17:26.072187 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:17:26.072187 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:17:26.072187 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:17:26.072187 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:17:26.073904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:17:26.073904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 04 01:17:26.073904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 04 01:17:26.073904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:17:26.073904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:17:26.073904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:17:26.073904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:17:26.073904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 134e1ff6-ab5a-41a8-9afb-8891869494bf could not be found. Sep 04 01:17:26.073904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:17:26.073904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:17:26.073904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:17:26.073904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:17:26.073904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:17:26.073904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:17:26.073904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:17:26.073904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:17:26.073904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 134e1ff6-ab5a-41a8-9afb-8891869494bf could not be found. Sep 04 01:17:26.073904 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:17:26.075422 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-9f42e361-9208-4e65-a9e6-345bbd1a8d20 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Returning 404 to user: Instance 134e1ff6-ab5a-41a8-9afb-8891869494bf could not be found. {{(pid=121036) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:17:26.075422 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-9f42e361-9208-4e65-a9e6-345bbd1a8d20 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf" status: 404 len: 111 microversion: 2.1 time: 0.030987 Sep 04 01:17:26.075422 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 628/1229] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:17:26 2022] GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 04 01:17:26.166183 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-b9bade18-27bb-4f4d-83c2-c07ae36ba6f5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-682841931", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-436573754", "security_groups": [{"name": "tempest-secgroup-smoke-2067064205"}], "block_device_mapping_v2": [{"uuid": "bd96c9d6-6bd7-42bd-8b3d-5eebdc874f47", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "ca88993b-9210-4173-b062-c40cdf9e1da6"}]}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:17:26.277982 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.network.neutron [None req-b9bade18-27bb-4f4d-83c2-c07ae36ba6f5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] validate_networks() for [('ca88993b-9210-4173-b062-c40cdf9e1da6', None, None, None, None, None)] {{(pid=121037) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 04 01:17:26.739982 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-b9bade18-27bb-4f4d-83c2-c07ae36ba6f5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Flavor limits 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 04 01:17:26.740400 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-b9bade18-27bb-4f4d-83c2-c07ae36ba6f5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Image limits 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 04 01:17:26.740869 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-b9bade18-27bb-4f4d-83c2-c07ae36ba6f5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Flavor pref 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 04 01:17:26.741335 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-b9bade18-27bb-4f4d-83c2-c07ae36ba6f5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Image pref 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 04 01:17:26.745427 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-b9bade18-27bb-4f4d-83c2-c07ae36ba6f5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 04 01:17:26.750858 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b9bade18-27bb-4f4d-83c2-c07ae36ba6f5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:26.751331 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b9bade18-27bb-4f4d-83c2-c07ae36ba6f5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:26.751759 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b9bade18-27bb-4f4d-83c2-c07ae36ba6f5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:26.752479 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b9bade18-27bb-4f4d-83c2-c07ae36ba6f5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:26.752922 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b9bade18-27bb-4f4d-83c2-c07ae36ba6f5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:26.753375 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b9bade18-27bb-4f4d-83c2-c07ae36ba6f5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:26.769688 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.quota [None req-b9bade18-27bb-4f4d-83c2-c07ae36ba6f5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Getting quotas for project 2506b96ac13541e6a7a47c5aedca67e5. Resources: {'ram', 'instances', 'cores'} {{(pid=121037) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 04 01:17:26.773516 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.quota [None req-b9bade18-27bb-4f4d-83c2-c07ae36ba6f5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Getting quotas for user 48c4f0c78a364c00a793666eeb5a520c and project 2506b96ac13541e6a7a47c5aedca67e5. Resources: {'ram', 'instances', 'cores'} {{(pid=121037) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 04 01:17:26.829217 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-b9bade18-27bb-4f4d-83c2-c07ae36ba6f5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Going to run 1 instances... {{(pid=121037) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 04 01:17:26.831334 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-b9bade18-27bb-4f4d-83c2-c07ae36ba6f5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 77c3060f-d328-4221-864f-8af5a9baae4c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='bd96c9d6-6bd7-42bd-8b3d-5eebdc874f47',volume_size=None,volume_type=None)] {{(pid=121037) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 04 01:17:27.262648 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-b9bade18-27bb-4f4d-83c2-c07ae36ba6f5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.100115 Sep 04 01:17:27.263376 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 602/1230] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 4 01:17:26 2022] POST /compute/v2.1/servers => generated 406 bytes in 1101 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 04 01:17:27.274991 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-fccd49b6-8dd9-4389-bc09-da934aeb7dec tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:17:27.275628 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-fccd49b6-8dd9-4389-bc09-da934aeb7dec tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 77c3060f-d328-4221-864f-8af5a9baae4c] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:17:27.330360 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-fccd49b6-8dd9-4389-bc09-da934aeb7dec tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:17:27.337178 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-fccd49b6-8dd9-4389-bc09-da934aeb7dec tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c" status: 200 len: 1093 microversion: 2.1 time: 0.066070 Sep 04 01:17:27.340340 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 629/1231] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:17:27 2022] GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c => generated 1093 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:17:28.360950 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-0e400e7c-aa20-4860-a3f3-39695e64bb79 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:17:28.361649 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-0e400e7c-aa20-4860-a3f3-39695e64bb79 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 77c3060f-d328-4221-864f-8af5a9baae4c] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:17:28.367774 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0e400e7c-aa20-4860-a3f3-39695e64bb79 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:28.368261 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0e400e7c-aa20-4860-a3f3-39695e64bb79 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:28.368689 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0e400e7c-aa20-4860-a3f3-39695e64bb79 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:28.369421 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0e400e7c-aa20-4860-a3f3-39695e64bb79 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:28.369873 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0e400e7c-aa20-4860-a3f3-39695e64bb79 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:28.374950 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0e400e7c-aa20-4860-a3f3-39695e64bb79 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:28.432055 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-0e400e7c-aa20-4860-a3f3-39695e64bb79 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:17:28.439908 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-0e400e7c-aa20-4860-a3f3-39695e64bb79 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c" status: 200 len: 1143 microversion: 2.1 time: 0.083856 Sep 04 01:17:28.441180 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 603/1232] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:17:28 2022] GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c => generated 1143 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:17:29.463179 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-cca31511-4625-43b3-b495-1d77795725ef tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:17:29.464635 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-cca31511-4625-43b3-b495-1d77795725ef tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 77c3060f-d328-4221-864f-8af5a9baae4c] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:17:29.472419 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cca31511-4625-43b3-b495-1d77795725ef tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:29.472943 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cca31511-4625-43b3-b495-1d77795725ef tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:29.473418 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cca31511-4625-43b3-b495-1d77795725ef tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:29.474127 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cca31511-4625-43b3-b495-1d77795725ef tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:29.474614 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cca31511-4625-43b3-b495-1d77795725ef tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:29.475275 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cca31511-4625-43b3-b495-1d77795725ef tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:29.541841 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-cca31511-4625-43b3-b495-1d77795725ef tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:17:29.550380 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-cca31511-4625-43b3-b495-1d77795725ef tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c" status: 200 len: 1199 microversion: 2.1 time: 0.091540 Sep 04 01:17:29.550959 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 630/1233] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:17:29 2022] GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c => generated 1199 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:17:30.574588 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-11162448-4c8f-4032-b01f-4b0b170057f8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:17:30.575273 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-11162448-4c8f-4032-b01f-4b0b170057f8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 77c3060f-d328-4221-864f-8af5a9baae4c] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:17:30.582673 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-11162448-4c8f-4032-b01f-4b0b170057f8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:30.583654 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-11162448-4c8f-4032-b01f-4b0b170057f8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:30.584224 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-11162448-4c8f-4032-b01f-4b0b170057f8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:30.585277 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-11162448-4c8f-4032-b01f-4b0b170057f8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:30.586141 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-11162448-4c8f-4032-b01f-4b0b170057f8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:30.587168 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-11162448-4c8f-4032-b01f-4b0b170057f8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:30.745418 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-11162448-4c8f-4032-b01f-4b0b170057f8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:17:30.753802 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-11162448-4c8f-4032-b01f-4b0b170057f8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c" status: 200 len: 1277 microversion: 2.1 time: 0.183144 Sep 04 01:17:30.754456 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 604/1234] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:17:30 2022] GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c => generated 1277 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:17:31.071598 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-e9b2bd26-1e59-4cef-8bc6-852c958a6c03 req-96002a9e-9841-44e2-a753-045b6e1325a2 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "77c3060f-d328-4221-864f-8af5a9baae4c", "tag": "da467063-2304-41a7-80d8-d19cc5f69f7a"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:17:31.081842 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-e9b2bd26-1e59-4cef-8bc6-852c958a6c03 req-96002a9e-9841-44e2-a753-045b6e1325a2 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:31.082360 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-e9b2bd26-1e59-4cef-8bc6-852c958a6c03 req-96002a9e-9841-44e2-a753-045b6e1325a2 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:31.082744 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-e9b2bd26-1e59-4cef-8bc6-852c958a6c03 req-96002a9e-9841-44e2-a753-045b6e1325a2 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:31.094130 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-e9b2bd26-1e59-4cef-8bc6-852c958a6c03 req-96002a9e-9841-44e2-a753-045b6e1325a2 service nova] Creating event network-changed:da467063-2304-41a7-80d8-d19cc5f69f7a for instance 77c3060f-d328-4221-864f-8af5a9baae4c on hv-cinder-maste Sep 04 01:17:31.099039 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-e9b2bd26-1e59-4cef-8bc6-852c958a6c03 req-96002a9e-9841-44e2-a753-045b6e1325a2 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031183 Sep 04 01:17:31.099633 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 631/1235] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:17:31 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:17:31.776919 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-52ab0159-c1be-45e5-be27-ea4784ae3c91 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:17:31.777551 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-52ab0159-c1be-45e5-be27-ea4784ae3c91 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 77c3060f-d328-4221-864f-8af5a9baae4c] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:17:31.784861 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-52ab0159-c1be-45e5-be27-ea4784ae3c91 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:31.785365 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-52ab0159-c1be-45e5-be27-ea4784ae3c91 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:31.785796 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-52ab0159-c1be-45e5-be27-ea4784ae3c91 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:31.786491 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-52ab0159-c1be-45e5-be27-ea4784ae3c91 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:31.786939 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-52ab0159-c1be-45e5-be27-ea4784ae3c91 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:31.787359 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-52ab0159-c1be-45e5-be27-ea4784ae3c91 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:31.936905 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-52ab0159-c1be-45e5-be27-ea4784ae3c91 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:17:31.945304 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-52ab0159-c1be-45e5-be27-ea4784ae3c91 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c" status: 200 len: 1265 microversion: 2.1 time: 0.172275 Sep 04 01:17:31.945872 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 605/1236] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:17:31 2022] GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c => generated 1265 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:17:32.969770 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-dcd98f8a-ae7d-41f2-b88a-157f5961c566 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:17:32.970466 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-dcd98f8a-ae7d-41f2-b88a-157f5961c566 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 77c3060f-d328-4221-864f-8af5a9baae4c] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:17:32.976817 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dcd98f8a-ae7d-41f2-b88a-157f5961c566 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:32.977333 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dcd98f8a-ae7d-41f2-b88a-157f5961c566 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:32.977767 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dcd98f8a-ae7d-41f2-b88a-157f5961c566 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:32.978455 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dcd98f8a-ae7d-41f2-b88a-157f5961c566 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:32.978910 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dcd98f8a-ae7d-41f2-b88a-157f5961c566 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:32.979335 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-dcd98f8a-ae7d-41f2-b88a-157f5961c566 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:33.110141 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-dcd98f8a-ae7d-41f2-b88a-157f5961c566 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:17:33.117936 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-dcd98f8a-ae7d-41f2-b88a-157f5961c566 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c" status: 200 len: 1265 microversion: 2.1 time: 0.153678 Sep 04 01:17:33.118497 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 632/1237] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:17:32 2022] GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c => generated 1265 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:17:34.141862 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-2fd4998c-b00f-4131-b1a8-4b2f2f87d2f5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:17:34.142462 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-2fd4998c-b00f-4131-b1a8-4b2f2f87d2f5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 77c3060f-d328-4221-864f-8af5a9baae4c] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:17:34.148670 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2fd4998c-b00f-4131-b1a8-4b2f2f87d2f5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:34.149172 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2fd4998c-b00f-4131-b1a8-4b2f2f87d2f5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:34.149611 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2fd4998c-b00f-4131-b1a8-4b2f2f87d2f5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:34.150338 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2fd4998c-b00f-4131-b1a8-4b2f2f87d2f5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:34.150795 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2fd4998c-b00f-4131-b1a8-4b2f2f87d2f5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:34.151228 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-2fd4998c-b00f-4131-b1a8-4b2f2f87d2f5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:34.282390 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-2fd4998c-b00f-4131-b1a8-4b2f2f87d2f5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:17:34.290038 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-2fd4998c-b00f-4131-b1a8-4b2f2f87d2f5 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c" status: 200 len: 1265 microversion: 2.1 time: 0.152101 Sep 04 01:17:34.295870 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 606/1238] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:17:34 2022] GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c => generated 1265 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:17:35.311426 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-4878b8f1-4b7e-49c6-8938-5f2581f56f2b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:17:35.312571 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-4878b8f1-4b7e-49c6-8938-5f2581f56f2b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 77c3060f-d328-4221-864f-8af5a9baae4c] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:17:35.319218 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4878b8f1-4b7e-49c6-8938-5f2581f56f2b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:35.319699 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4878b8f1-4b7e-49c6-8938-5f2581f56f2b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:35.320133 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4878b8f1-4b7e-49c6-8938-5f2581f56f2b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:35.320812 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4878b8f1-4b7e-49c6-8938-5f2581f56f2b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:35.321296 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4878b8f1-4b7e-49c6-8938-5f2581f56f2b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:35.321717 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4878b8f1-4b7e-49c6-8938-5f2581f56f2b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:35.450894 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-4878b8f1-4b7e-49c6-8938-5f2581f56f2b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:17:35.458923 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-4878b8f1-4b7e-49c6-8938-5f2581f56f2b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c" status: 200 len: 1265 microversion: 2.1 time: 0.151369 Sep 04 01:17:35.459486 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 633/1239] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:17:35 2022] GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c => generated 1265 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:17:36.481522 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-34d61d10-9ddf-44da-9eab-4a0382195fcf tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:17:36.482156 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-34d61d10-9ddf-44da-9eab-4a0382195fcf tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 77c3060f-d328-4221-864f-8af5a9baae4c] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:17:36.488806 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-34d61d10-9ddf-44da-9eab-4a0382195fcf tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:36.489322 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-34d61d10-9ddf-44da-9eab-4a0382195fcf tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:36.489750 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-34d61d10-9ddf-44da-9eab-4a0382195fcf tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:36.490431 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-34d61d10-9ddf-44da-9eab-4a0382195fcf tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:36.490873 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-34d61d10-9ddf-44da-9eab-4a0382195fcf tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:36.491289 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-34d61d10-9ddf-44da-9eab-4a0382195fcf tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:36.620025 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-34d61d10-9ddf-44da-9eab-4a0382195fcf tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:17:36.630028 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-34d61d10-9ddf-44da-9eab-4a0382195fcf tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c" status: 200 len: 1265 microversion: 2.1 time: 0.152277 Sep 04 01:17:36.630671 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 607/1240] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:17:36 2022] GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c => generated 1265 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:17:37.654464 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-4364fc9e-d6e3-4535-ba6c-912b8580f650 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:17:37.655016 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-4364fc9e-d6e3-4535-ba6c-912b8580f650 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 77c3060f-d328-4221-864f-8af5a9baae4c] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:17:37.658321 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell0:SELECT=2 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:17:37.663476 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4364fc9e-d6e3-4535-ba6c-912b8580f650 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:37.666653 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4364fc9e-d6e3-4535-ba6c-912b8580f650 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:37.666653 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4364fc9e-d6e3-4535-ba6c-912b8580f650 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:37.666653 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4364fc9e-d6e3-4535-ba6c-912b8580f650 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:37.666653 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4364fc9e-d6e3-4535-ba6c-912b8580f650 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:37.667130 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-4364fc9e-d6e3-4535-ba6c-912b8580f650 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:37.800673 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-4364fc9e-d6e3-4535-ba6c-912b8580f650 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:17:37.808369 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-4364fc9e-d6e3-4535-ba6c-912b8580f650 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c" status: 200 len: 1265 microversion: 2.1 time: 0.157929 Sep 04 01:17:37.808929 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 634/1241] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:17:37 2022] GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c => generated 1265 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:17:38.831993 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-8831467e-1349-4bde-b9b3-7223ad525983 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:17:38.832598 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-8831467e-1349-4bde-b9b3-7223ad525983 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 77c3060f-d328-4221-864f-8af5a9baae4c] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:17:38.834477 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell0:SELECT=2 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:17:38.835327 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell0:SELECT=6 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:17:38.840543 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8831467e-1349-4bde-b9b3-7223ad525983 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:38.841020 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8831467e-1349-4bde-b9b3-7223ad525983 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:38.841487 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8831467e-1349-4bde-b9b3-7223ad525983 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:38.842168 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8831467e-1349-4bde-b9b3-7223ad525983 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:38.842616 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8831467e-1349-4bde-b9b3-7223ad525983 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:38.843253 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8831467e-1349-4bde-b9b3-7223ad525983 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:38.983742 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-8831467e-1349-4bde-b9b3-7223ad525983 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:17:38.992305 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-8831467e-1349-4bde-b9b3-7223ad525983 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c" status: 200 len: 1265 microversion: 2.1 time: 0.164771 Sep 04 01:17:38.992873 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 608/1242] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:17:38 2022] GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c => generated 1265 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:17:40.016346 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-9b849ddd-858f-4ada-a97e-92c4ec7ae05b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:17:40.016951 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-9b849ddd-858f-4ada-a97e-92c4ec7ae05b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 77c3060f-d328-4221-864f-8af5a9baae4c] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:17:40.023395 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9b849ddd-858f-4ada-a97e-92c4ec7ae05b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:40.023871 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9b849ddd-858f-4ada-a97e-92c4ec7ae05b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:40.024304 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9b849ddd-858f-4ada-a97e-92c4ec7ae05b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:40.025109 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9b849ddd-858f-4ada-a97e-92c4ec7ae05b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:40.025569 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9b849ddd-858f-4ada-a97e-92c4ec7ae05b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:40.025989 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9b849ddd-858f-4ada-a97e-92c4ec7ae05b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:40.154590 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-9b849ddd-858f-4ada-a97e-92c4ec7ae05b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:17:40.162530 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-9b849ddd-858f-4ada-a97e-92c4ec7ae05b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c" status: 200 len: 1265 microversion: 2.1 time: 0.150654 Sep 04 01:17:40.163125 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 635/1243] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:17:40 2022] GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c => generated 1265 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:17:41.185631 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-5dc2715f-0204-44a9-908e-d3491425b6ed tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:17:41.187267 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-5dc2715f-0204-44a9-908e-d3491425b6ed tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 77c3060f-d328-4221-864f-8af5a9baae4c] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:17:41.197192 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5dc2715f-0204-44a9-908e-d3491425b6ed tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:41.197655 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5dc2715f-0204-44a9-908e-d3491425b6ed tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:41.198091 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5dc2715f-0204-44a9-908e-d3491425b6ed tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:41.198834 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5dc2715f-0204-44a9-908e-d3491425b6ed tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:41.199281 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5dc2715f-0204-44a9-908e-d3491425b6ed tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:41.199698 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-5dc2715f-0204-44a9-908e-d3491425b6ed tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:41.321984 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-dd5e3bac-d2a7-4895-95a3-ff7bc80573e5 req-9f288d69-13e2-4f74-959e-6f398935ba43 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "77c3060f-d328-4221-864f-8af5a9baae4c", "name": "network-vif-plugged", "status": "completed", "tag": "da467063-2304-41a7-80d8-d19cc5f69f7a"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:17:41.333650 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-dd5e3bac-d2a7-4895-95a3-ff7bc80573e5 req-9f288d69-13e2-4f74-959e-6f398935ba43 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:41.334396 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-dd5e3bac-d2a7-4895-95a3-ff7bc80573e5 req-9f288d69-13e2-4f74-959e-6f398935ba43 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:41.335115 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-dd5e3bac-d2a7-4895-95a3-ff7bc80573e5 req-9f288d69-13e2-4f74-959e-6f398935ba43 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:41.346656 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-dd5e3bac-d2a7-4895-95a3-ff7bc80573e5 req-9f288d69-13e2-4f74-959e-6f398935ba43 service nova] Creating event network-vif-plugged:da467063-2304-41a7-80d8-d19cc5f69f7a for instance 77c3060f-d328-4221-864f-8af5a9baae4c on hv-cinder-maste Sep 04 01:17:41.352382 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-dd5e3bac-d2a7-4895-95a3-ff7bc80573e5 req-9f288d69-13e2-4f74-959e-6f398935ba43 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.171628 Sep 04 01:17:41.352980 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 609/1244] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:17:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:17:41.357896 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-5dc2715f-0204-44a9-908e-d3491425b6ed tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:17:41.366420 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-5dc2715f-0204-44a9-908e-d3491425b6ed tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c" status: 200 len: 1265 microversion: 2.1 time: 0.184699 Sep 04 01:17:41.366983 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 636/1245] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:17:41 2022] GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c => generated 1265 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:17:42.391402 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-f4a181e6-e6ee-494d-ae67-c962ebbf51a4 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:17:42.392096 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-f4a181e6-e6ee-494d-ae67-c962ebbf51a4 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 77c3060f-d328-4221-864f-8af5a9baae4c] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:17:42.398362 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f4a181e6-e6ee-494d-ae67-c962ebbf51a4 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:42.398844 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f4a181e6-e6ee-494d-ae67-c962ebbf51a4 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:42.399277 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f4a181e6-e6ee-494d-ae67-c962ebbf51a4 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:42.399966 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f4a181e6-e6ee-494d-ae67-c962ebbf51a4 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:42.400426 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f4a181e6-e6ee-494d-ae67-c962ebbf51a4 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:42.400843 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-f4a181e6-e6ee-494d-ae67-c962ebbf51a4 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:42.536467 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-f4a181e6-e6ee-494d-ae67-c962ebbf51a4 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:17:42.544342 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-f4a181e6-e6ee-494d-ae67-c962ebbf51a4 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c" status: 200 len: 1265 microversion: 2.1 time: 0.158188 Sep 04 01:17:42.544882 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 610/1246] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:17:42 2022] GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c => generated 1265 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:17:43.567912 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-9d46c23f-4a5d-4461-89de-837377b99043 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:17:43.568553 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-9d46c23f-4a5d-4461-89de-837377b99043 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 77c3060f-d328-4221-864f-8af5a9baae4c] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:17:43.574728 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9d46c23f-4a5d-4461-89de-837377b99043 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:43.575208 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9d46c23f-4a5d-4461-89de-837377b99043 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:43.575646 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9d46c23f-4a5d-4461-89de-837377b99043 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:43.576331 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9d46c23f-4a5d-4461-89de-837377b99043 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:43.576786 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9d46c23f-4a5d-4461-89de-837377b99043 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:43.577870 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-9d46c23f-4a5d-4461-89de-837377b99043 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:43.729602 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-9d46c23f-4a5d-4461-89de-837377b99043 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:17:43.737375 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-9d46c23f-4a5d-4461-89de-837377b99043 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c" status: 200 len: 1451 microversion: 2.1 time: 0.173345 Sep 04 01:17:43.737946 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 637/1247] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:17:43 2022] GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c => generated 1451 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:17:43.759942 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-d6c99cf6-1ae1-46ff-9dd5-26e93ba26b25 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:17:43.760546 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-d6c99cf6-1ae1-46ff-9dd5-26e93ba26b25 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 77c3060f-d328-4221-864f-8af5a9baae4c] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:17:43.767628 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d6c99cf6-1ae1-46ff-9dd5-26e93ba26b25 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:43.768106 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d6c99cf6-1ae1-46ff-9dd5-26e93ba26b25 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:43.768589 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d6c99cf6-1ae1-46ff-9dd5-26e93ba26b25 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:43.769299 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d6c99cf6-1ae1-46ff-9dd5-26e93ba26b25 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:43.769741 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d6c99cf6-1ae1-46ff-9dd5-26e93ba26b25 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:43.770164 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-d6c99cf6-1ae1-46ff-9dd5-26e93ba26b25 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:43.898883 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-d6c99cf6-1ae1-46ff-9dd5-26e93ba26b25 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:17:43.907614 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-d6c99cf6-1ae1-46ff-9dd5-26e93ba26b25 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c" status: 200 len: 1451 microversion: 2.1 time: 0.151539 Sep 04 01:17:43.908179 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 611/1248] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:17:43 2022] GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c => generated 1451 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:17:46.070635 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-dc63d22e-96ad-446f-9a72-438f8a4e4cf8 req-fdfff50c-be31-48ad-bc87-5bf2d04fc812 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "77c3060f-d328-4221-864f-8af5a9baae4c", "tag": "da467063-2304-41a7-80d8-d19cc5f69f7a"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:17:46.081203 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-dc63d22e-96ad-446f-9a72-438f8a4e4cf8 req-fdfff50c-be31-48ad-bc87-5bf2d04fc812 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:17:46.083078 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-dc63d22e-96ad-446f-9a72-438f8a4e4cf8 req-fdfff50c-be31-48ad-bc87-5bf2d04fc812 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:17:46.083078 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-dc63d22e-96ad-446f-9a72-438f8a4e4cf8 req-fdfff50c-be31-48ad-bc87-5bf2d04fc812 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:17:46.094908 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-dc63d22e-96ad-446f-9a72-438f8a4e4cf8 req-fdfff50c-be31-48ad-bc87-5bf2d04fc812 service nova] Creating event network-changed:da467063-2304-41a7-80d8-d19cc5f69f7a for instance 77c3060f-d328-4221-864f-8af5a9baae4c on hv-cinder-maste Sep 04 01:17:46.100258 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-dc63d22e-96ad-446f-9a72-438f8a4e4cf8 req-fdfff50c-be31-48ad-bc87-5bf2d04fc812 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033054 Sep 04 01:17:46.101147 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 638/1249] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:17:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:19:21.736062 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_api:SELECT=41,nova_api:INSERT=3 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:19:21.737012 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=72 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:19:21.737903 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 04 01:19:21.738076 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Sun Sep 4 01:19:21 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.42) !!! Sep 04 01:19:21.744049 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-67b151ab-ae91-4db7-8a54-693c9152a657 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-321780702", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-436573754", "security_groups": [{"name": "tempest-secgroup-smoke-2067064205"}], "block_device_mapping_v2": [{"uuid": "759cca84-ae61-41f6-a5a8-cae7a3644af9", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "ca88993b-9210-4173-b062-c40cdf9e1da6"}]}} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:19:21.910783 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.network.neutron [None req-67b151ab-ae91-4db7-8a54-693c9152a657 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] validate_networks() for [('ca88993b-9210-4173-b062-c40cdf9e1da6', None, None, None, None, None)] {{(pid=121037) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 04 01:19:22.471071 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-67b151ab-ae91-4db7-8a54-693c9152a657 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Flavor limits 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 04 01:19:22.471498 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-67b151ab-ae91-4db7-8a54-693c9152a657 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Image limits 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 04 01:19:22.471985 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-67b151ab-ae91-4db7-8a54-693c9152a657 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Flavor pref 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 04 01:19:22.472525 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-67b151ab-ae91-4db7-8a54-693c9152a657 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Image pref 0:0:0 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 04 01:19:22.472972 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.virt.hardware [None req-67b151ab-ae91-4db7-8a54-693c9152a657 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=121037) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 04 01:19:22.481773 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-67b151ab-ae91-4db7-8a54-693c9152a657 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:22.482251 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-67b151ab-ae91-4db7-8a54-693c9152a657 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:22.482688 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-67b151ab-ae91-4db7-8a54-693c9152a657 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:22.483470 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-67b151ab-ae91-4db7-8a54-693c9152a657 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:22.483925 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-67b151ab-ae91-4db7-8a54-693c9152a657 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:22.484352 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-67b151ab-ae91-4db7-8a54-693c9152a657 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:22.501816 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.quota [None req-67b151ab-ae91-4db7-8a54-693c9152a657 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Getting quotas for project 2506b96ac13541e6a7a47c5aedca67e5. Resources: {'ram', 'instances', 'cores'} {{(pid=121037) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 04 01:19:22.506575 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.quota [None req-67b151ab-ae91-4db7-8a54-693c9152a657 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Getting quotas for user 48c4f0c78a364c00a793666eeb5a520c and project 2506b96ac13541e6a7a47c5aedca67e5. Resources: {'ram', 'instances', 'cores'} {{(pid=121037) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 04 01:19:22.562815 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-67b151ab-ae91-4db7-8a54-693c9152a657 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Going to run 1 instances... {{(pid=121037) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 04 01:19:22.565537 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-67b151ab-ae91-4db7-8a54-693c9152a657 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 1af95d46-3118-449e-9a37-63298088e3ea] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='759cca84-ae61-41f6-a5a8-cae7a3644af9',volume_size=None,volume_type=None)] {{(pid=121037) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 04 01:19:22.937658 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-67b151ab-ae91-4db7-8a54-693c9152a657 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.203302 Sep 04 01:19:22.940713 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 612/1250] 10.222.0.42 () {60 vars in 1147 bytes} [Sun Sep 4 01:19:21 2022] POST /compute/v2.1/servers => generated 406 bytes in 1205 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 04 01:19:22.948171 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_api:SELECT=35,nova_api:UPDATE=1 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:19:22.949009 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=70,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:19:22.954323 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-f98854c3-1913-4321-bfdb-e919a24b53e8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:19:22.955037 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-f98854c3-1913-4321-bfdb-e919a24b53e8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 1af95d46-3118-449e-9a37-63298088e3ea] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:19:23.010124 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-f98854c3-1913-4321-bfdb-e919a24b53e8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:19:23.017012 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-f98854c3-1913-4321-bfdb-e919a24b53e8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea" status: 200 len: 1093 microversion: 2.1 time: 0.071073 Sep 04 01:19:23.017605 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 639/1251] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:19:22 2022] GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea => generated 1093 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:19:24.041511 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-50f03cdd-55fa-49bd-8653-6cc4faa351e9 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:19:24.042122 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-50f03cdd-55fa-49bd-8653-6cc4faa351e9 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 1af95d46-3118-449e-9a37-63298088e3ea] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:19:24.101599 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-50f03cdd-55fa-49bd-8653-6cc4faa351e9 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:19:24.108704 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-50f03cdd-55fa-49bd-8653-6cc4faa351e9 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea" status: 200 len: 1093 microversion: 2.1 time: 0.071847 Sep 04 01:19:24.109106 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 613/1252] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:19:24 2022] GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea => generated 1093 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 04 01:19:25.139584 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-dfd15f2b-d759-4318-8498-dea3930f4743 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:19:25.140192 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-dfd15f2b-d759-4318-8498-dea3930f4743 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 1af95d46-3118-449e-9a37-63298088e3ea] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:19:25.197330 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-dfd15f2b-d759-4318-8498-dea3930f4743 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:19:25.204831 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-dfd15f2b-d759-4318-8498-dea3930f4743 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea" status: 200 len: 1093 microversion: 2.1 time: 0.069387 Sep 04 01:19:25.207945 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 640/1253] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:19:25 2022] GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea => generated 1093 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:19:26.228117 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-36e7bc76-3cda-4208-919c-c6f506dba9ab tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:19:26.228705 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-36e7bc76-3cda-4208-919c-c6f506dba9ab tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 1af95d46-3118-449e-9a37-63298088e3ea] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:19:26.280093 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-36e7bc76-3cda-4208-919c-c6f506dba9ab tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:19:26.287667 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-36e7bc76-3cda-4208-919c-c6f506dba9ab tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea" status: 200 len: 1093 microversion: 2.1 time: 0.063410 Sep 04 01:19:26.288230 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 614/1254] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:19:26 2022] GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea => generated 1093 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:19:27.310128 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-bbcd149b-2ef9-4f1d-a6a8-51d9af952a35 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:19:27.310750 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-bbcd149b-2ef9-4f1d-a6a8-51d9af952a35 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 1af95d46-3118-449e-9a37-63298088e3ea] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:19:27.318434 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bbcd149b-2ef9-4f1d-a6a8-51d9af952a35 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:27.318929 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bbcd149b-2ef9-4f1d-a6a8-51d9af952a35 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:27.319358 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bbcd149b-2ef9-4f1d-a6a8-51d9af952a35 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:27.320055 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bbcd149b-2ef9-4f1d-a6a8-51d9af952a35 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:27.320509 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bbcd149b-2ef9-4f1d-a6a8-51d9af952a35 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:27.320939 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bbcd149b-2ef9-4f1d-a6a8-51d9af952a35 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:27.380875 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-bbcd149b-2ef9-4f1d-a6a8-51d9af952a35 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:19:27.388469 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-bbcd149b-2ef9-4f1d-a6a8-51d9af952a35 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea" status: 200 len: 1143 microversion: 2.1 time: 0.082285 Sep 04 01:19:27.389037 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 641/1255] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:19:27 2022] GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea => generated 1143 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:19:28.411704 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-eb2271db-7c3d-442c-992d-165b1b0f1904 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:19:28.412331 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-eb2271db-7c3d-442c-992d-165b1b0f1904 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 1af95d46-3118-449e-9a37-63298088e3ea] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:19:28.420361 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-eb2271db-7c3d-442c-992d-165b1b0f1904 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:28.420844 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-eb2271db-7c3d-442c-992d-165b1b0f1904 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:28.421891 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-eb2271db-7c3d-442c-992d-165b1b0f1904 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:28.422835 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-eb2271db-7c3d-442c-992d-165b1b0f1904 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:28.423293 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-eb2271db-7c3d-442c-992d-165b1b0f1904 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:28.423732 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-eb2271db-7c3d-442c-992d-165b1b0f1904 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:28.493341 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-eb2271db-7c3d-442c-992d-165b1b0f1904 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:19:28.501492 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-eb2271db-7c3d-442c-992d-165b1b0f1904 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea" status: 200 len: 1191 microversion: 2.1 time: 0.093647 Sep 04 01:19:28.502056 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 615/1256] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:19:28 2022] GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea => generated 1191 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:19:29.525440 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-ce198d13-7f07-4bed-9f32-2227acbef841 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:19:29.526058 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-ce198d13-7f07-4bed-9f32-2227acbef841 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 1af95d46-3118-449e-9a37-63298088e3ea] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:19:29.532634 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ce198d13-7f07-4bed-9f32-2227acbef841 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:29.533179 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ce198d13-7f07-4bed-9f32-2227acbef841 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:29.533632 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ce198d13-7f07-4bed-9f32-2227acbef841 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:29.534338 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ce198d13-7f07-4bed-9f32-2227acbef841 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:29.534789 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ce198d13-7f07-4bed-9f32-2227acbef841 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:29.535221 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ce198d13-7f07-4bed-9f32-2227acbef841 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:29.598973 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-ce198d13-7f07-4bed-9f32-2227acbef841 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:19:29.606824 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-ce198d13-7f07-4bed-9f32-2227acbef841 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea" status: 200 len: 1209 microversion: 2.1 time: 0.085444 Sep 04 01:19:29.607460 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 642/1257] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:19:29 2022] GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea => generated 1209 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:19:30.630741 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-8eaa839c-4e8c-4f8a-bee9-af97a0ff7882 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:19:30.644261 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-8eaa839c-4e8c-4f8a-bee9-af97a0ff7882 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 1af95d46-3118-449e-9a37-63298088e3ea] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:19:30.644261 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8eaa839c-4e8c-4f8a-bee9-af97a0ff7882 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:30.644261 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8eaa839c-4e8c-4f8a-bee9-af97a0ff7882 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:30.644261 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8eaa839c-4e8c-4f8a-bee9-af97a0ff7882 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:30.646100 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8eaa839c-4e8c-4f8a-bee9-af97a0ff7882 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:30.646100 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8eaa839c-4e8c-4f8a-bee9-af97a0ff7882 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:30.646100 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8eaa839c-4e8c-4f8a-bee9-af97a0ff7882 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:30.785054 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-0c06a4f7-434c-48fe-b531-a46edd1545f7 req-75b4a764-a40d-4db1-b1a9-822df344e362 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1af95d46-3118-449e-9a37-63298088e3ea", "tag": "7013b090-a02a-4f55-8e96-e743cde4d73a"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:19:30.786495 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-8eaa839c-4e8c-4f8a-bee9-af97a0ff7882 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:19:30.795607 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-8eaa839c-4e8c-4f8a-bee9-af97a0ff7882 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea" status: 200 len: 1277 microversion: 2.1 time: 0.169114 Sep 04 01:19:30.797650 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-0c06a4f7-434c-48fe-b531-a46edd1545f7 req-75b4a764-a40d-4db1-b1a9-822df344e362 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:30.797797 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 616/1258] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:19:30 2022] GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea => generated 1277 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:19:30.797914 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-0c06a4f7-434c-48fe-b531-a46edd1545f7 req-75b4a764-a40d-4db1-b1a9-822df344e362 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:30.797914 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-0c06a4f7-434c-48fe-b531-a46edd1545f7 req-75b4a764-a40d-4db1-b1a9-822df344e362 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:30.810214 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-0c06a4f7-434c-48fe-b531-a46edd1545f7 req-75b4a764-a40d-4db1-b1a9-822df344e362 service nova] Creating event network-changed:7013b090-a02a-4f55-8e96-e743cde4d73a for instance 1af95d46-3118-449e-9a37-63298088e3ea on hv-cinder-maste Sep 04 01:19:30.815210 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-0c06a4f7-434c-48fe-b531-a46edd1545f7 req-75b4a764-a40d-4db1-b1a9-822df344e362 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034899 Sep 04 01:19:30.820588 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 643/1259] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:19:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:19:31.818808 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-b09854ab-d4d4-476e-86b9-d1239baf4c74 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:19:31.819457 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-b09854ab-d4d4-476e-86b9-d1239baf4c74 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 1af95d46-3118-449e-9a37-63298088e3ea] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:19:31.825951 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b09854ab-d4d4-476e-86b9-d1239baf4c74 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:31.826433 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b09854ab-d4d4-476e-86b9-d1239baf4c74 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:31.826869 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b09854ab-d4d4-476e-86b9-d1239baf4c74 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:31.827561 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b09854ab-d4d4-476e-86b9-d1239baf4c74 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:31.828015 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b09854ab-d4d4-476e-86b9-d1239baf4c74 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:31.828441 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b09854ab-d4d4-476e-86b9-d1239baf4c74 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:31.962238 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-b09854ab-d4d4-476e-86b9-d1239baf4c74 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:19:31.969988 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-b09854ab-d4d4-476e-86b9-d1239baf4c74 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea" status: 200 len: 1265 microversion: 2.1 time: 0.155051 Sep 04 01:19:31.970500 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 617/1260] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:19:31 2022] GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea => generated 1265 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:19:32.994498 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b95d2426-8ffe-48e3-8c16-f3e1deebfc8d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:19:32.995187 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-b95d2426-8ffe-48e3-8c16-f3e1deebfc8d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 1af95d46-3118-449e-9a37-63298088e3ea] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:19:33.002164 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b95d2426-8ffe-48e3-8c16-f3e1deebfc8d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:33.002642 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b95d2426-8ffe-48e3-8c16-f3e1deebfc8d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:33.003093 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b95d2426-8ffe-48e3-8c16-f3e1deebfc8d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:33.003780 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b95d2426-8ffe-48e3-8c16-f3e1deebfc8d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:33.004949 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b95d2426-8ffe-48e3-8c16-f3e1deebfc8d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:33.005453 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b95d2426-8ffe-48e3-8c16-f3e1deebfc8d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:33.136837 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-b95d2426-8ffe-48e3-8c16-f3e1deebfc8d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:19:33.144449 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-b95d2426-8ffe-48e3-8c16-f3e1deebfc8d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea" status: 200 len: 1265 microversion: 2.1 time: 0.154296 Sep 04 01:19:33.145590 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 644/1261] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:19:32 2022] GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea => generated 1265 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:19:34.167707 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-62963568-8d44-45e3-ba7e-d08a36c8a742 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:19:34.168350 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-62963568-8d44-45e3-ba7e-d08a36c8a742 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 1af95d46-3118-449e-9a37-63298088e3ea] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:19:34.170214 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell0:SELECT=2 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:19:34.175001 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-62963568-8d44-45e3-ba7e-d08a36c8a742 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:34.175481 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-62963568-8d44-45e3-ba7e-d08a36c8a742 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:34.175985 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-62963568-8d44-45e3-ba7e-d08a36c8a742 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:34.177331 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-62963568-8d44-45e3-ba7e-d08a36c8a742 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:34.178166 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-62963568-8d44-45e3-ba7e-d08a36c8a742 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:34.179745 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-62963568-8d44-45e3-ba7e-d08a36c8a742 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:34.308699 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-62963568-8d44-45e3-ba7e-d08a36c8a742 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:19:34.316273 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-62963568-8d44-45e3-ba7e-d08a36c8a742 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea" status: 200 len: 1265 microversion: 2.1 time: 0.152596 Sep 04 01:19:34.316964 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 618/1262] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:19:34 2022] GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea => generated 1265 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:19:35.339301 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-bd68f0cd-de52-47ac-9e4d-b1bc271d632d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:19:35.339899 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-bd68f0cd-de52-47ac-9e4d-b1bc271d632d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 1af95d46-3118-449e-9a37-63298088e3ea] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:19:35.342372 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell0:SELECT=4 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:19:35.347001 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bd68f0cd-de52-47ac-9e4d-b1bc271d632d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:35.357850 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bd68f0cd-de52-47ac-9e4d-b1bc271d632d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:35.357850 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bd68f0cd-de52-47ac-9e4d-b1bc271d632d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:35.357850 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bd68f0cd-de52-47ac-9e4d-b1bc271d632d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:35.357850 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bd68f0cd-de52-47ac-9e4d-b1bc271d632d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:35.358285 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-bd68f0cd-de52-47ac-9e4d-b1bc271d632d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:35.483213 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-bd68f0cd-de52-47ac-9e4d-b1bc271d632d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:19:35.494423 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-bd68f0cd-de52-47ac-9e4d-b1bc271d632d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea" status: 200 len: 1265 microversion: 2.1 time: 0.158988 Sep 04 01:19:35.494994 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 645/1263] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:19:35 2022] GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea => generated 1265 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:19:36.517584 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-415f9cc1-1e88-43ed-84e2-ba02211336f8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:19:36.518194 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-415f9cc1-1e88-43ed-84e2-ba02211336f8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 1af95d46-3118-449e-9a37-63298088e3ea] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:19:36.521175 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell0:SELECT=10 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:19:36.525949 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-415f9cc1-1e88-43ed-84e2-ba02211336f8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:36.526428 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-415f9cc1-1e88-43ed-84e2-ba02211336f8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:36.526868 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-415f9cc1-1e88-43ed-84e2-ba02211336f8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:36.527548 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-415f9cc1-1e88-43ed-84e2-ba02211336f8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:36.527998 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-415f9cc1-1e88-43ed-84e2-ba02211336f8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:36.528423 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-415f9cc1-1e88-43ed-84e2-ba02211336f8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:36.659576 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-415f9cc1-1e88-43ed-84e2-ba02211336f8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:19:36.667391 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-415f9cc1-1e88-43ed-84e2-ba02211336f8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea" status: 200 len: 1265 microversion: 2.1 time: 0.153635 Sep 04 01:19:36.667962 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 619/1264] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:19:36 2022] GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea => generated 1265 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:19:37.690233 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-d1c1c0f7-c51c-4fc6-81d1-5658325d21eb tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:19:37.690841 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-d1c1c0f7-c51c-4fc6-81d1-5658325d21eb tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 1af95d46-3118-449e-9a37-63298088e3ea] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:19:37.697501 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d1c1c0f7-c51c-4fc6-81d1-5658325d21eb tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:37.697977 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d1c1c0f7-c51c-4fc6-81d1-5658325d21eb tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:37.698409 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d1c1c0f7-c51c-4fc6-81d1-5658325d21eb tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:37.699088 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d1c1c0f7-c51c-4fc6-81d1-5658325d21eb tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:37.699535 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d1c1c0f7-c51c-4fc6-81d1-5658325d21eb tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:37.699953 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-d1c1c0f7-c51c-4fc6-81d1-5658325d21eb tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:37.829596 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-d1c1c0f7-c51c-4fc6-81d1-5658325d21eb tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:19:37.837201 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-d1c1c0f7-c51c-4fc6-81d1-5658325d21eb tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea" status: 200 len: 1265 microversion: 2.1 time: 0.150786 Sep 04 01:19:37.837763 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 646/1265] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:19:37 2022] GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea => generated 1265 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:19:38.860408 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-144dd081-924c-4095-a760-e13d1bd4faa9 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:19:38.861023 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-144dd081-924c-4095-a760-e13d1bd4faa9 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 1af95d46-3118-449e-9a37-63298088e3ea] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:19:38.867570 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-144dd081-924c-4095-a760-e13d1bd4faa9 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:38.868049 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-144dd081-924c-4095-a760-e13d1bd4faa9 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:38.868504 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-144dd081-924c-4095-a760-e13d1bd4faa9 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:38.869216 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-144dd081-924c-4095-a760-e13d1bd4faa9 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:38.869674 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-144dd081-924c-4095-a760-e13d1bd4faa9 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:38.870098 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-144dd081-924c-4095-a760-e13d1bd4faa9 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:39.000857 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-144dd081-924c-4095-a760-e13d1bd4faa9 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:19:39.009445 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-144dd081-924c-4095-a760-e13d1bd4faa9 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea" status: 200 len: 1265 microversion: 2.1 time: 0.152908 Sep 04 01:19:39.010055 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 620/1266] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:19:38 2022] GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea => generated 1265 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:19:40.031710 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-cda759e5-0490-40dd-822d-d0599ce2311b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:19:40.032258 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-cda759e5-0490-40dd-822d-d0599ce2311b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 1af95d46-3118-449e-9a37-63298088e3ea] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:19:40.038762 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cda759e5-0490-40dd-822d-d0599ce2311b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:40.039232 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cda759e5-0490-40dd-822d-d0599ce2311b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:40.039661 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cda759e5-0490-40dd-822d-d0599ce2311b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:40.040337 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cda759e5-0490-40dd-822d-d0599ce2311b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:40.040782 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cda759e5-0490-40dd-822d-d0599ce2311b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:40.041238 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-cda759e5-0490-40dd-822d-d0599ce2311b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:40.173988 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-cda759e5-0490-40dd-822d-d0599ce2311b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:19:40.182316 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-cda759e5-0490-40dd-822d-d0599ce2311b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea" status: 200 len: 1265 microversion: 2.1 time: 0.154482 Sep 04 01:19:40.182889 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 647/1267] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:19:40 2022] GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea => generated 1265 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:19:41.256491 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-0d008c4e-9184-49eb-8e7f-895515040795 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:19:41.257134 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-0d008c4e-9184-49eb-8e7f-895515040795 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 1af95d46-3118-449e-9a37-63298088e3ea] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:19:41.263032 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0d008c4e-9184-49eb-8e7f-895515040795 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:41.263517 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0d008c4e-9184-49eb-8e7f-895515040795 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:41.263955 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0d008c4e-9184-49eb-8e7f-895515040795 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:41.264646 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0d008c4e-9184-49eb-8e7f-895515040795 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:41.265121 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0d008c4e-9184-49eb-8e7f-895515040795 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:41.265551 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0d008c4e-9184-49eb-8e7f-895515040795 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:41.397232 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-0d008c4e-9184-49eb-8e7f-895515040795 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:19:41.404941 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-0d008c4e-9184-49eb-8e7f-895515040795 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea" status: 200 len: 1265 microversion: 2.1 time: 0.203481 Sep 04 01:19:41.405822 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 621/1268] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:19:41 2022] GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea => generated 1265 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:19:41.736614 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-2d67f0f2-b3ca-4312-a0a4-b36eef611d56 req-284623ca-2066-4d7f-a794-cb99980f93e4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1af95d46-3118-449e-9a37-63298088e3ea", "name": "network-vif-plugged", "status": "completed", "tag": "7013b090-a02a-4f55-8e96-e743cde4d73a"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:19:41.747910 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-2d67f0f2-b3ca-4312-a0a4-b36eef611d56 req-284623ca-2066-4d7f-a794-cb99980f93e4 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:41.748368 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-2d67f0f2-b3ca-4312-a0a4-b36eef611d56 req-284623ca-2066-4d7f-a794-cb99980f93e4 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:41.748791 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-2d67f0f2-b3ca-4312-a0a4-b36eef611d56 req-284623ca-2066-4d7f-a794-cb99980f93e4 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:41.759565 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-2d67f0f2-b3ca-4312-a0a4-b36eef611d56 req-284623ca-2066-4d7f-a794-cb99980f93e4 service nova] Creating event network-vif-plugged:7013b090-a02a-4f55-8e96-e743cde4d73a for instance 1af95d46-3118-449e-9a37-63298088e3ea on hv-cinder-maste Sep 04 01:19:41.764665 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-2d67f0f2-b3ca-4312-a0a4-b36eef611d56 req-284623ca-2066-4d7f-a794-cb99980f93e4 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033799 Sep 04 01:19:41.765321 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 648/1269] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:19:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 04 01:19:42.431007 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-0225e5cd-41c8-4b9a-ac8a-edab6363d12d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:19:42.431560 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-0225e5cd-41c8-4b9a-ac8a-edab6363d12d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 1af95d46-3118-449e-9a37-63298088e3ea] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:19:42.437987 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0225e5cd-41c8-4b9a-ac8a-edab6363d12d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:42.438464 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0225e5cd-41c8-4b9a-ac8a-edab6363d12d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:42.438899 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0225e5cd-41c8-4b9a-ac8a-edab6363d12d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:42.439589 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0225e5cd-41c8-4b9a-ac8a-edab6363d12d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:42.440042 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0225e5cd-41c8-4b9a-ac8a-edab6363d12d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:42.440466 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-0225e5cd-41c8-4b9a-ac8a-edab6363d12d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:42.569432 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-0225e5cd-41c8-4b9a-ac8a-edab6363d12d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:19:42.577285 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-0225e5cd-41c8-4b9a-ac8a-edab6363d12d tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea" status: 200 len: 1265 microversion: 2.1 time: 0.150225 Sep 04 01:19:42.577852 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 622/1270] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:19:42 2022] GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea => generated 1265 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:19:43.601356 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-06d9b86c-d68a-43c0-95ba-9b961786d84b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:19:43.602056 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-06d9b86c-d68a-43c0-95ba-9b961786d84b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 1af95d46-3118-449e-9a37-63298088e3ea] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:19:43.608255 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-06d9b86c-d68a-43c0-95ba-9b961786d84b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:43.608733 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-06d9b86c-d68a-43c0-95ba-9b961786d84b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:43.609202 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-06d9b86c-d68a-43c0-95ba-9b961786d84b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:43.609920 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-06d9b86c-d68a-43c0-95ba-9b961786d84b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:43.610373 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-06d9b86c-d68a-43c0-95ba-9b961786d84b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:43.610800 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-06d9b86c-d68a-43c0-95ba-9b961786d84b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:43.740670 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-06d9b86c-d68a-43c0-95ba-9b961786d84b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:19:43.748794 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-06d9b86c-d68a-43c0-95ba-9b961786d84b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea" status: 200 len: 1452 microversion: 2.1 time: 0.152658 Sep 04 01:19:43.749399 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 649/1271] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:19:43 2022] GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea => generated 1452 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:19:43.776725 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-70848ac7-39aa-4bc9-addb-b9108a66fb73 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:19:43.777437 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-70848ac7-39aa-4bc9-addb-b9108a66fb73 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 1af95d46-3118-449e-9a37-63298088e3ea] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:19:43.783839 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-70848ac7-39aa-4bc9-addb-b9108a66fb73 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:43.784317 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-70848ac7-39aa-4bc9-addb-b9108a66fb73 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:43.784749 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-70848ac7-39aa-4bc9-addb-b9108a66fb73 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:43.786002 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-70848ac7-39aa-4bc9-addb-b9108a66fb73 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:43.787116 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-70848ac7-39aa-4bc9-addb-b9108a66fb73 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:43.787546 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-70848ac7-39aa-4bc9-addb-b9108a66fb73 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:43.922344 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-70848ac7-39aa-4bc9-addb-b9108a66fb73 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:19:43.930628 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-70848ac7-39aa-4bc9-addb-b9108a66fb73 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea" status: 200 len: 1452 microversion: 2.1 time: 0.157950 Sep 04 01:19:43.931199 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 623/1272] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:19:43 2022] GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea => generated 1452 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:19:46.097786 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-773375e1-eace-4cf6-98d0-716eb89c6495 req-5be2f0e2-e9eb-4359-93b0-8fae31eafc4a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1af95d46-3118-449e-9a37-63298088e3ea", "tag": "7013b090-a02a-4f55-8e96-e743cde4d73a"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:19:46.107838 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-773375e1-eace-4cf6-98d0-716eb89c6495 req-5be2f0e2-e9eb-4359-93b0-8fae31eafc4a service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:19:46.108312 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-773375e1-eace-4cf6-98d0-716eb89c6495 req-5be2f0e2-e9eb-4359-93b0-8fae31eafc4a service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:19:46.108736 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-773375e1-eace-4cf6-98d0-716eb89c6495 req-5be2f0e2-e9eb-4359-93b0-8fae31eafc4a service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:19:46.120021 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-773375e1-eace-4cf6-98d0-716eb89c6495 req-5be2f0e2-e9eb-4359-93b0-8fae31eafc4a service nova] Creating event network-changed:7013b090-a02a-4f55-8e96-e743cde4d73a for instance 1af95d46-3118-449e-9a37-63298088e3ea on hv-cinder-maste Sep 04 01:19:46.125209 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-773375e1-eace-4cf6-98d0-716eb89c6495 req-5be2f0e2-e9eb-4359-93b0-8fae31eafc4a service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031078 Sep 04 01:19:46.126064 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 650/1273] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:19:46 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:21:14.292405 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_api:SELECT=41,nova_api:INSERT=3 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:21:14.296393 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG dbcounter [-] [121037] Writing DB stats nova_cell1:SELECT=56 {{(pid=121037) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:21:14.296393 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 04 01:21:14.296393 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Sun Sep 4 01:21:14 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea (ip 10.222.0.42) !!! Sep 04 01:21:14.296873 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_api:SELECT=28 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:21:14.298171 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG dbcounter [-] [121036] Writing DB stats nova_cell1:SELECT=48 {{(pid=121036) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 04 01:21:14.299064 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 04 01:21:14.299204 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Sun Sep 4 01:21:14 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.222.0.42) !!! Sep 04 01:21:14.299905 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-66de674b-6720-4620-9fc4-611e51e5c685 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:21:14.300406 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-66de674b-6720-4620-9fc4-611e51e5c685 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 1af95d46-3118-449e-9a37-63298088e3ea] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:21:14.302128 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 04 01:21:14.302284 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Sun Sep 4 01:21:14 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.222.0.42) !!! Sep 04 01:21:14.308441 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-66de674b-6720-4620-9fc4-611e51e5c685 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:21:14.308970 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-66de674b-6720-4620-9fc4-611e51e5c685 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:21:14.309437 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-66de674b-6720-4620-9fc4-611e51e5c685 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:21:14.310407 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-66de674b-6720-4620-9fc4-611e51e5c685 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:21:14.310867 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-66de674b-6720-4620-9fc4-611e51e5c685 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:21:14.311298 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-66de674b-6720-4620-9fc4-611e51e5c685 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:21:14.339814 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-66de674b-6720-4620-9fc4-611e51e5c685 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 1af95d46-3118-449e-9a37-63298088e3ea] Going to try to terminate instance {{(pid=121037) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 04 01:21:14.389915 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-4c0fc854-0ff5-40c2-90e5-841475516af5 req-5283bb31-219e-431c-aea0-21c263c6e573 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1af95d46-3118-449e-9a37-63298088e3ea", "tag": "7013b090-a02a-4f55-8e96-e743cde4d73a"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:21:14.401326 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-4c0fc854-0ff5-40c2-90e5-841475516af5 req-5283bb31-219e-431c-aea0-21c263c6e573 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:21:14.401851 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-4c0fc854-0ff5-40c2-90e5-841475516af5 req-5283bb31-219e-431c-aea0-21c263c6e573 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:21:14.402297 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-4c0fc854-0ff5-40c2-90e5-841475516af5 req-5283bb31-219e-431c-aea0-21c263c6e573 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:21:14.413950 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-4c0fc854-0ff5-40c2-90e5-841475516af5 req-5283bb31-219e-431c-aea0-21c263c6e573 service nova] Creating event network-changed:7013b090-a02a-4f55-8e96-e743cde4d73a for instance 1af95d46-3118-449e-9a37-63298088e3ea on hv-cinder-maste Sep 04 01:21:14.419402 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-4c0fc854-0ff5-40c2-90e5-841475516af5 req-5283bb31-219e-431c-aea0-21c263c6e573 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.126763 Sep 04 01:21:14.420169 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 651/1274] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:21:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:21:14.633636 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-66de674b-6720-4620-9fc4-611e51e5c685 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "DELETE /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea" status: 204 len: 0 microversion: 2.1 time: 0.343013 Sep 04 01:21:14.637769 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 624/1275] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 4 01:21:14 2022] DELETE /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea => generated 0 bytes in 345 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 04 01:21:14.645592 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-434c559d-2380-480c-b1f9-3a705dcd3545 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:21:14.646520 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-434c559d-2380-480c-b1f9-3a705dcd3545 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 1af95d46-3118-449e-9a37-63298088e3ea] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:21:14.653814 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-434c559d-2380-480c-b1f9-3a705dcd3545 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:21:14.654310 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-434c559d-2380-480c-b1f9-3a705dcd3545 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:21:14.654903 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-434c559d-2380-480c-b1f9-3a705dcd3545 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:21:14.655849 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-434c559d-2380-480c-b1f9-3a705dcd3545 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:21:14.656432 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-434c559d-2380-480c-b1f9-3a705dcd3545 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:21:14.657010 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-434c559d-2380-480c-b1f9-3a705dcd3545 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:21:14.864741 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-434c559d-2380-480c-b1f9-3a705dcd3545 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:21:14.873261 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-434c559d-2380-480c-b1f9-3a705dcd3545 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea" status: 200 len: 1577 microversion: 2.1 time: 0.233155 Sep 04 01:21:14.873837 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 652/1276] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:21:14 2022] GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea => generated 1577 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:21:15.898225 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-31752679-e305-4a81-bc21-fe5cfdac74e2 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:21:15.898915 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-31752679-e305-4a81-bc21-fe5cfdac74e2 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 1af95d46-3118-449e-9a37-63298088e3ea] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:21:15.905477 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-31752679-e305-4a81-bc21-fe5cfdac74e2 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:21:15.905990 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-31752679-e305-4a81-bc21-fe5cfdac74e2 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:21:15.906426 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-31752679-e305-4a81-bc21-fe5cfdac74e2 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:21:15.907111 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-31752679-e305-4a81-bc21-fe5cfdac74e2 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:21:15.907559 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-31752679-e305-4a81-bc21-fe5cfdac74e2 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:21:15.907981 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-31752679-e305-4a81-bc21-fe5cfdac74e2 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:21:16.102104 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-31752679-e305-4a81-bc21-fe5cfdac74e2 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:21:16.109954 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-31752679-e305-4a81-bc21-fe5cfdac74e2 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea" status: 200 len: 1577 microversion: 2.1 time: 0.216040 Sep 04 01:21:16.110520 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 625/1277] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:21:15 2022] GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea => generated 1577 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:21:16.869754 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [req-9422e89f-d752-47a5-889e-cf5a9ef345a5 req-3a70c1ba-82f2-4262-b4be-afcbdfe72cd8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1af95d46-3118-449e-9a37-63298088e3ea", "name": "network-vif-deleted", "tag": "7013b090-a02a-4f55-8e96-e743cde4d73a"}]} {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:21:16.881804 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-9422e89f-d752-47a5-889e-cf5a9ef345a5 req-3a70c1ba-82f2-4262-b4be-afcbdfe72cd8 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:21:16.886099 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-9422e89f-d752-47a5-889e-cf5a9ef345a5 req-3a70c1ba-82f2-4262-b4be-afcbdfe72cd8 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:21:16.886099 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [req-9422e89f-d752-47a5-889e-cf5a9ef345a5 req-3a70c1ba-82f2-4262-b4be-afcbdfe72cd8 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:21:16.897117 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.compute.server_external_events [req-9422e89f-d752-47a5-889e-cf5a9ef345a5 req-3a70c1ba-82f2-4262-b4be-afcbdfe72cd8 service nova] Creating event network-vif-deleted:7013b090-a02a-4f55-8e96-e743cde4d73a for instance 1af95d46-3118-449e-9a37-63298088e3ea on hv-cinder-maste Sep 04 01:21:16.903911 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [req-9422e89f-d752-47a5-889e-cf5a9ef345a5 req-3a70c1ba-82f2-4262-b4be-afcbdfe72cd8 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037822 Sep 04 01:21:16.904510 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 653/1278] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:21:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:21:17.134063 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-1f88ea53-c4b2-4f60-9c1e-422d332d27c6 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:21:17.134675 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-1f88ea53-c4b2-4f60-9c1e-422d332d27c6 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 1af95d46-3118-449e-9a37-63298088e3ea] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:21:17.141344 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1f88ea53-c4b2-4f60-9c1e-422d332d27c6 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:21:17.141832 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1f88ea53-c4b2-4f60-9c1e-422d332d27c6 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:21:17.142262 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1f88ea53-c4b2-4f60-9c1e-422d332d27c6 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:21:17.142955 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1f88ea53-c4b2-4f60-9c1e-422d332d27c6 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:21:17.143405 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1f88ea53-c4b2-4f60-9c1e-422d332d27c6 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:21:17.143829 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-1f88ea53-c4b2-4f60-9c1e-422d332d27c6 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:21:17.208946 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-1f88ea53-c4b2-4f60-9c1e-422d332d27c6 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:21:17.217140 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-1f88ea53-c4b2-4f60-9c1e-422d332d27c6 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea" status: 200 len: 1224 microversion: 2.1 time: 0.086728 Sep 04 01:21:17.217728 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 626/1279] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:21:17 2022] GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea => generated 1224 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:21:18.240790 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-685c9ccc-d60a-44f7-8d68-3c399e6c2e17 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:21:18.241536 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-685c9ccc-d60a-44f7-8d68-3c399e6c2e17 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 1af95d46-3118-449e-9a37-63298088e3ea] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:21:18.248365 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-685c9ccc-d60a-44f7-8d68-3c399e6c2e17 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:21:18.248852 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-685c9ccc-d60a-44f7-8d68-3c399e6c2e17 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:21:18.249308 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-685c9ccc-d60a-44f7-8d68-3c399e6c2e17 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:21:18.250005 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-685c9ccc-d60a-44f7-8d68-3c399e6c2e17 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:21:18.250459 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-685c9ccc-d60a-44f7-8d68-3c399e6c2e17 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:21:18.250907 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-685c9ccc-d60a-44f7-8d68-3c399e6c2e17 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:21:18.282591 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-685c9ccc-d60a-44f7-8d68-3c399e6c2e17 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:21:18.283060 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-685c9ccc-d60a-44f7-8d68-3c399e6c2e17 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:21:18.283490 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-685c9ccc-d60a-44f7-8d68-3c399e6c2e17 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:21:18.283955 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.objects.instance [None req-685c9ccc-d60a-44f7-8d68-3c399e6c2e17 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lazy-loading 'fault' on Instance uuid 1af95d46-3118-449e-9a37-63298088e3ea {{(pid=121036) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:21:18.329672 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-685c9ccc-d60a-44f7-8d68-3c399e6c2e17 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:21:18.338296 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-685c9ccc-d60a-44f7-8d68-3c399e6c2e17 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea" status: 200 len: 1229 microversion: 2.1 time: 0.101938 Sep 04 01:21:18.339084 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 654/1280] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:21:18 2022] GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea => generated 1229 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:21:19.362679 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-45199de1-172e-4e9a-9e4b-8242c88de362 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:21:19.363291 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-45199de1-172e-4e9a-9e4b-8242c88de362 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 1af95d46-3118-449e-9a37-63298088e3ea] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:21:19.369884 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-45199de1-172e-4e9a-9e4b-8242c88de362 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:21:19.370360 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-45199de1-172e-4e9a-9e4b-8242c88de362 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:21:19.370790 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-45199de1-172e-4e9a-9e4b-8242c88de362 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:21:19.371502 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-45199de1-172e-4e9a-9e4b-8242c88de362 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:21:19.371950 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-45199de1-172e-4e9a-9e4b-8242c88de362 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:21:19.372375 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-45199de1-172e-4e9a-9e4b-8242c88de362 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:21:19.385958 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.wsgi [None req-45199de1-172e-4e9a-9e4b-8242c88de362 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] HTTP exception thrown: Instance 1af95d46-3118-449e-9a37-63298088e3ea could not be found. Sep 04 01:21:19.387570 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-45199de1-172e-4e9a-9e4b-8242c88de362 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Request method failure captured: Sep 04 01:21:19.387570 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: request: GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea HTTP/1.1 Sep 04 01:21:19.387570 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept: application/json Sep 04 01:21:19.387570 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept-Encoding: identity Sep 04 01:21:19.387570 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Connection: close Sep 04 01:21:19.387570 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Length: 0 Sep 04 01:21:19.387570 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Type: application/json Sep 04 01:21:19.387570 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Host: 10.222.0.42 Sep 04 01:21:19.387570 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Openstack-System-Scope: None Sep 04 01:21:19.387570 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: User-Agent: python-urllib3/1.26.12 Sep 04 01:21:19.387570 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Auth-Token: gAAAAABjE_v3xFr83GF5HAJz0F1ePckFApHoM5uGMmIbR4HzFsfIZD_EnhL2OGYcBnvrJRXBe6apIZ4muzHym6lqQRHT8CWNei8sBt1BevsxBM3QKX99qXYG3ttnjXaWqmmKrkT5XoKcl347t24GOiR5KkDxLGu96o2UlmXDRT33-ZG56RntQ2w Sep 04 01:21:19.387570 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Id: None Sep 04 01:21:19.387570 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Name: None Sep 04 01:21:19.387570 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Identity-Status: Confirmed Sep 04 01:21:19.387570 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Is-Admin-Project: True Sep 04 01:21:19.387570 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Id: default Sep 04 01:21:19.387570 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Name: Default Sep 04 01:21:19.387570 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Id: 2506b96ac13541e6a7a47c5aedca67e5 Sep 04 01:21:19.387570 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Name: tempest-TestVolumeBootPattern-1801805144 Sep 04 01:21:19.387570 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Role: reader,member Sep 04 01:21:19.387570 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Roles: reader,member Sep 04 01:21:19.389484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2506b96ac13541e6a7a47c5aedca67e5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_2506b96ac13541e6a7a47c5aedca67e5", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2506b96ac13541e6a7a47c5aedca67e5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/2506b96ac13541e6a7a47c5aedca67e5"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:21:19.389484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant: tempest-TestVolumeBootPattern-1801805144 Sep 04 01:21:19.389484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Id: 2506b96ac13541e6a7a47c5aedca67e5 Sep 04 01:21:19.389484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Name: tempest-TestVolumeBootPattern-1801805144 Sep 04 01:21:19.389484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User: tempest-TestVolumeBootPattern-1801805144-project Sep 04 01:21:19.389484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Id: default Sep 04 01:21:19.389484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Name: Default Sep 04 01:21:19.389484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Id: 48c4f0c78a364c00a793666eeb5a520c Sep 04 01:21:19.389484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Name: tempest-TestVolumeBootPattern-1801805144-project Sep 04 01:21:19.389484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: method: > Sep 04 01:21:19.389484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: exception: Instance 1af95d46-3118-449e-9a37-63298088e3ea could not be found. Sep 04 01:21:19.389484 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: action_args: {'id': '1af95d46-3118-449e-9a37-63298088e3ea'} Sep 04 01:21:19.390514 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:21:19.390514 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:21:19.390514 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:21:19.390514 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:21:19.390514 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:21:19.390514 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:21:19.390514 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:21:19.390514 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:21:19.390514 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:21:19.390514 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:21:19.390514 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1af95d46-3118-449e-9a37-63298088e3ea could not be found. Sep 04 01:21:19.390514 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:21:19.390514 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:21:19.390514 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:21:19.390514 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:21:19.390514 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:21:19.390514 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:21:19.390514 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:21:19.390514 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:21:19.390514 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:21:19.392204 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:21:19.392204 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 04 01:21:19.392204 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 04 01:21:19.392204 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:21:19.392204 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:21:19.392204 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:21:19.392204 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:21:19.392204 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1af95d46-3118-449e-9a37-63298088e3ea could not be found. Sep 04 01:21:19.392204 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:21:19.392204 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:21:19.392204 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:21:19.392204 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:21:19.392204 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:21:19.392204 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:21:19.392204 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:21:19.392204 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:21:19.392204 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1af95d46-3118-449e-9a37-63298088e3ea could not be found. Sep 04 01:21:19.392204 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:21:19.393793 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-45199de1-172e-4e9a-9e4b-8242c88de362 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Returning 404 to user: Instance 1af95d46-3118-449e-9a37-63298088e3ea could not be found. {{(pid=121037) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:21:19.393793 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-45199de1-172e-4e9a-9e4b-8242c88de362 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea" status: 404 len: 111 microversion: 2.1 time: 0.030140 Sep 04 01:21:19.393793 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 627/1281] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:21:19 2022] GET /compute/v2.1/servers/1af95d46-3118-449e-9a37-63298088e3ea => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 04 01:21:22.770290 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-164cc562-4cf1-4a1f-a9c4-c542c527e0e1 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:21:22.771702 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-164cc562-4cf1-4a1f-a9c4-c542c527e0e1 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 77c3060f-d328-4221-864f-8af5a9baae4c] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:21:22.772347 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-2cb6cdb5-e723-4bb9-aea0-9244a1e08310 req-d064aca1-0c76-4bb2-b1a8-96bedc4fb670 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "77c3060f-d328-4221-864f-8af5a9baae4c", "tag": "da467063-2304-41a7-80d8-d19cc5f69f7a"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:21:22.780055 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-164cc562-4cf1-4a1f-a9c4-c542c527e0e1 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:21:22.780840 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-164cc562-4cf1-4a1f-a9c4-c542c527e0e1 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:21:22.781528 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-164cc562-4cf1-4a1f-a9c4-c542c527e0e1 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:21:22.782452 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-164cc562-4cf1-4a1f-a9c4-c542c527e0e1 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:21:22.782598 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-2cb6cdb5-e723-4bb9-aea0-9244a1e08310 req-d064aca1-0c76-4bb2-b1a8-96bedc4fb670 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:21:22.782909 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-164cc562-4cf1-4a1f-a9c4-c542c527e0e1 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:21:22.783083 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-2cb6cdb5-e723-4bb9-aea0-9244a1e08310 req-d064aca1-0c76-4bb2-b1a8-96bedc4fb670 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:21:22.783400 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-2cb6cdb5-e723-4bb9-aea0-9244a1e08310 req-d064aca1-0c76-4bb2-b1a8-96bedc4fb670 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:21:22.783619 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-164cc562-4cf1-4a1f-a9c4-c542c527e0e1 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:21:22.795389 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-2cb6cdb5-e723-4bb9-aea0-9244a1e08310 req-d064aca1-0c76-4bb2-b1a8-96bedc4fb670 service nova] Creating event network-changed:da467063-2304-41a7-80d8-d19cc5f69f7a for instance 77c3060f-d328-4221-864f-8af5a9baae4c on hv-cinder-maste Sep 04 01:21:22.803724 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-2cb6cdb5-e723-4bb9-aea0-9244a1e08310 req-d064aca1-0c76-4bb2-b1a8-96bedc4fb670 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.041196 Sep 04 01:21:22.804339 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 628/1282] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:21:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:21:22.810697 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-164cc562-4cf1-4a1f-a9c4-c542c527e0e1 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 77c3060f-d328-4221-864f-8af5a9baae4c] Going to try to terminate instance {{(pid=121036) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 04 01:21:23.070723 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-164cc562-4cf1-4a1f-a9c4-c542c527e0e1 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "DELETE /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c" status: 204 len: 0 microversion: 2.1 time: 0.310332 Sep 04 01:21:23.071631 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 655/1283] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 4 01:21:22 2022] DELETE /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c => generated 0 bytes in 313 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 04 01:21:23.080123 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-8ea9ff5d-fcb2-4810-96fb-9f5999922151 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:21:23.080813 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-8ea9ff5d-fcb2-4810-96fb-9f5999922151 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 77c3060f-d328-4221-864f-8af5a9baae4c] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:21:23.088201 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8ea9ff5d-fcb2-4810-96fb-9f5999922151 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:21:23.089050 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8ea9ff5d-fcb2-4810-96fb-9f5999922151 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:21:23.089511 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8ea9ff5d-fcb2-4810-96fb-9f5999922151 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:21:23.090425 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8ea9ff5d-fcb2-4810-96fb-9f5999922151 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:21:23.090893 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8ea9ff5d-fcb2-4810-96fb-9f5999922151 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:21:23.091305 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-8ea9ff5d-fcb2-4810-96fb-9f5999922151 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:21:23.225048 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-8ea9ff5d-fcb2-4810-96fb-9f5999922151 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:21:23.232862 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-8ea9ff5d-fcb2-4810-96fb-9f5999922151 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c" status: 200 len: 1575 microversion: 2.1 time: 0.156879 Sep 04 01:21:23.233459 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 629/1284] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:21:23 2022] GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c => generated 1575 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:21:24.262507 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-b9ff1220-7d1b-4559-aff5-27d60e9adba8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:21:24.263305 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-b9ff1220-7d1b-4559-aff5-27d60e9adba8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 77c3060f-d328-4221-864f-8af5a9baae4c] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:21:24.270031 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b9ff1220-7d1b-4559-aff5-27d60e9adba8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:21:24.270629 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b9ff1220-7d1b-4559-aff5-27d60e9adba8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:21:24.271473 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b9ff1220-7d1b-4559-aff5-27d60e9adba8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:21:24.272162 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b9ff1220-7d1b-4559-aff5-27d60e9adba8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:21:24.272617 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b9ff1220-7d1b-4559-aff5-27d60e9adba8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:21:24.273045 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-b9ff1220-7d1b-4559-aff5-27d60e9adba8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:21:24.503241 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-b9ff1220-7d1b-4559-aff5-27d60e9adba8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:21:24.511257 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-b9ff1220-7d1b-4559-aff5-27d60e9adba8 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c" status: 200 len: 1575 microversion: 2.1 time: 0.254151 Sep 04 01:21:24.514641 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 656/1285] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:21:24 2022] GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c => generated 1575 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:21:25.342543 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [req-5d2af239-f66a-434c-8ac5-db7420484524 req-25d8ed68-7ba9-4aab-b2cd-2a98b1e2f892 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "77c3060f-d328-4221-864f-8af5a9baae4c", "name": "network-vif-deleted", "tag": "da467063-2304-41a7-80d8-d19cc5f69f7a"}]} {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 04 01:21:25.355191 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-5d2af239-f66a-434c-8ac5-db7420484524 req-25d8ed68-7ba9-4aab-b2cd-2a98b1e2f892 service nova] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:21:25.355656 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-5d2af239-f66a-434c-8ac5-db7420484524 req-25d8ed68-7ba9-4aab-b2cd-2a98b1e2f892 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:21:25.356076 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [req-5d2af239-f66a-434c-8ac5-db7420484524 req-25d8ed68-7ba9-4aab-b2cd-2a98b1e2f892 service nova] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:21:25.369786 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.compute.server_external_events [req-5d2af239-f66a-434c-8ac5-db7420484524 req-25d8ed68-7ba9-4aab-b2cd-2a98b1e2f892 service nova] Creating event network-vif-deleted:da467063-2304-41a7-80d8-d19cc5f69f7a for instance 77c3060f-d328-4221-864f-8af5a9baae4c on hv-cinder-maste Sep 04 01:21:25.376551 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [req-5d2af239-f66a-434c-8ac5-db7420484524 req-25d8ed68-7ba9-4aab-b2cd-2a98b1e2f892 service nova] 10.222.0.42 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038524 Sep 04 01:21:25.377184 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 630/1286] 10.222.0.42 () {64 vars in 1317 bytes} [Sun Sep 4 01:21:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 04 01:21:25.535164 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-906a3fbf-6d90-4a29-ba25-00ac363918b3 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:21:25.535836 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-906a3fbf-6d90-4a29-ba25-00ac363918b3 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 77c3060f-d328-4221-864f-8af5a9baae4c] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:21:25.543440 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-906a3fbf-6d90-4a29-ba25-00ac363918b3 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:21:25.544384 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-906a3fbf-6d90-4a29-ba25-00ac363918b3 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:21:25.544384 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-906a3fbf-6d90-4a29-ba25-00ac363918b3 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:21:25.545051 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-906a3fbf-6d90-4a29-ba25-00ac363918b3 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:21:25.545533 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-906a3fbf-6d90-4a29-ba25-00ac363918b3 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:21:25.545960 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-906a3fbf-6d90-4a29-ba25-00ac363918b3 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:21:25.613858 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.policy [None req-906a3fbf-6d90-4a29-ba25-00ac363918b3 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121036) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:21:25.622567 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-906a3fbf-6d90-4a29-ba25-00ac363918b3 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c" status: 200 len: 1224 microversion: 2.1 time: 0.091389 Sep 04 01:21:25.623413 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 657/1287] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:21:25 2022] GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c => generated 1224 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:21:26.647190 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-b5aae36a-1f69-4e27-b912-bd060e164724 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:21:26.647814 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-b5aae36a-1f69-4e27-b912-bd060e164724 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 77c3060f-d328-4221-864f-8af5a9baae4c] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:21:26.654129 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b5aae36a-1f69-4e27-b912-bd060e164724 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:21:26.654673 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b5aae36a-1f69-4e27-b912-bd060e164724 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:21:26.655111 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b5aae36a-1f69-4e27-b912-bd060e164724 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:21:26.655796 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b5aae36a-1f69-4e27-b912-bd060e164724 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:21:26.656248 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b5aae36a-1f69-4e27-b912-bd060e164724 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:21:26.656696 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b5aae36a-1f69-4e27-b912-bd060e164724 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:21:26.690880 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b5aae36a-1f69-4e27-b912-bd060e164724 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:21:26.691362 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b5aae36a-1f69-4e27-b912-bd060e164724 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:21:26.691798 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-b5aae36a-1f69-4e27-b912-bd060e164724 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:21:26.692264 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.objects.instance [None req-b5aae36a-1f69-4e27-b912-bd060e164724 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lazy-loading 'fault' on Instance uuid 77c3060f-d328-4221-864f-8af5a9baae4c {{(pid=121037) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 04 01:21:26.736880 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.policy [None req-b5aae36a-1f69-4e27-b912-bd060e164724 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '48c4f0c78a364c00a793666eeb5a520c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2506b96ac13541e6a7a47c5aedca67e5', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=121037) authorize /opt/stack/nova/nova/policy.py:199}} Sep 04 01:21:26.744521 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-b5aae36a-1f69-4e27-b912-bd060e164724 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c" status: 200 len: 1183 microversion: 2.1 time: 0.102131 Sep 04 01:21:26.745102 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 631/1288] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:21:26 2022] GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c => generated 1183 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 04 01:21:27.768407 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-ec6cf6ce-d770-46ba-83b5-02ee45668e5b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:21:27.769051 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-ec6cf6ce-d770-46ba-83b5-02ee45668e5b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 77c3060f-d328-4221-864f-8af5a9baae4c] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:21:27.776269 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ec6cf6ce-d770-46ba-83b5-02ee45668e5b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:21:27.776746 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ec6cf6ce-d770-46ba-83b5-02ee45668e5b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:21:27.777222 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ec6cf6ce-d770-46ba-83b5-02ee45668e5b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:21:27.777911 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ec6cf6ce-d770-46ba-83b5-02ee45668e5b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:21:27.778361 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ec6cf6ce-d770-46ba-83b5-02ee45668e5b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:21:27.778786 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-ec6cf6ce-d770-46ba-83b5-02ee45668e5b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:21:27.792356 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.wsgi [None req-ec6cf6ce-d770-46ba-83b5-02ee45668e5b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] HTTP exception thrown: Instance 77c3060f-d328-4221-864f-8af5a9baae4c could not be found. Sep 04 01:21:27.794033 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-ec6cf6ce-d770-46ba-83b5-02ee45668e5b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Request method failure captured: Sep 04 01:21:27.794033 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: request: GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c HTTP/1.1 Sep 04 01:21:27.794033 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept: application/json Sep 04 01:21:27.794033 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept-Encoding: identity Sep 04 01:21:27.794033 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Connection: close Sep 04 01:21:27.794033 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Length: 0 Sep 04 01:21:27.794033 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Type: application/json Sep 04 01:21:27.794033 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Host: 10.222.0.42 Sep 04 01:21:27.794033 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Openstack-System-Scope: None Sep 04 01:21:27.794033 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: User-Agent: python-urllib3/1.26.12 Sep 04 01:21:27.794033 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Auth-Token: gAAAAABjE_v3xFr83GF5HAJz0F1ePckFApHoM5uGMmIbR4HzFsfIZD_EnhL2OGYcBnvrJRXBe6apIZ4muzHym6lqQRHT8CWNei8sBt1BevsxBM3QKX99qXYG3ttnjXaWqmmKrkT5XoKcl347t24GOiR5KkDxLGu96o2UlmXDRT33-ZG56RntQ2w Sep 04 01:21:27.794033 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Id: None Sep 04 01:21:27.794033 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Name: None Sep 04 01:21:27.794033 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Identity-Status: Confirmed Sep 04 01:21:27.794033 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Is-Admin-Project: True Sep 04 01:21:27.794033 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Id: default Sep 04 01:21:27.794033 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Name: Default Sep 04 01:21:27.794033 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Id: 2506b96ac13541e6a7a47c5aedca67e5 Sep 04 01:21:27.794033 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Name: tempest-TestVolumeBootPattern-1801805144 Sep 04 01:21:27.794033 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Role: reader,member Sep 04 01:21:27.794033 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Roles: reader,member Sep 04 01:21:27.795757 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2506b96ac13541e6a7a47c5aedca67e5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_2506b96ac13541e6a7a47c5aedca67e5", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2506b96ac13541e6a7a47c5aedca67e5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/2506b96ac13541e6a7a47c5aedca67e5"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:21:27.795757 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant: tempest-TestVolumeBootPattern-1801805144 Sep 04 01:21:27.795757 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Id: 2506b96ac13541e6a7a47c5aedca67e5 Sep 04 01:21:27.795757 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Name: tempest-TestVolumeBootPattern-1801805144 Sep 04 01:21:27.795757 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User: tempest-TestVolumeBootPattern-1801805144-project Sep 04 01:21:27.795757 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Id: default Sep 04 01:21:27.795757 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Name: Default Sep 04 01:21:27.795757 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Id: 48c4f0c78a364c00a793666eeb5a520c Sep 04 01:21:27.795757 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Name: tempest-TestVolumeBootPattern-1801805144-project Sep 04 01:21:27.795757 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: method: > Sep 04 01:21:27.795757 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: exception: Instance 77c3060f-d328-4221-864f-8af5a9baae4c could not be found. Sep 04 01:21:27.795757 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: action_args: {'id': '77c3060f-d328-4221-864f-8af5a9baae4c'} Sep 04 01:21:27.796866 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:21:27.796866 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:21:27.796866 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:21:27.796866 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:21:27.796866 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:21:27.796866 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:21:27.796866 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:21:27.796866 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:21:27.796866 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:21:27.796866 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:21:27.796866 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 77c3060f-d328-4221-864f-8af5a9baae4c could not be found. Sep 04 01:21:27.796866 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:21:27.796866 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:21:27.796866 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:21:27.796866 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:21:27.796866 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:21:27.796866 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:21:27.796866 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:21:27.796866 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:21:27.796866 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:21:27.798325 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:21:27.798325 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 04 01:21:27.798325 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 04 01:21:27.798325 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:21:27.798325 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:21:27.798325 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:21:27.798325 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:21:27.798325 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 77c3060f-d328-4221-864f-8af5a9baae4c could not be found. Sep 04 01:21:27.798325 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:21:27.798325 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:21:27.798325 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:21:27.798325 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:21:27.798325 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:21:27.798325 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:21:27.798325 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:21:27.798325 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:21:27.798325 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 77c3060f-d328-4221-864f-8af5a9baae4c could not be found. Sep 04 01:21:27.798325 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:21:27.799854 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-ec6cf6ce-d770-46ba-83b5-02ee45668e5b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Returning 404 to user: Instance 77c3060f-d328-4221-864f-8af5a9baae4c could not be found. {{(pid=121036) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:21:27.799854 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-ec6cf6ce-d770-46ba-83b5-02ee45668e5b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c" status: 404 len: 111 microversion: 2.1 time: 0.030976 Sep 04 01:21:27.799854 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 658/1289] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:21:27 2022] GET /compute/v2.1/servers/77c3060f-d328-4221-864f-8af5a9baae4c => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 04 01:21:28.307047 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-ea9bedb5-ec22-4cad-bc24-a848426c62a4 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:21:28.307571 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.compute.api [None req-ea9bedb5-ec22-4cad-bc24-a848426c62a4 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 134e1ff6-ab5a-41a8-9afb-8891869494bf] Fetching instance by UUID {{(pid=121037) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:21:28.313938 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea9bedb5-ec22-4cad-bc24-a848426c62a4 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:21:28.314414 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea9bedb5-ec22-4cad-bc24-a848426c62a4 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:21:28.314841 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea9bedb5-ec22-4cad-bc24-a848426c62a4 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:21:28.315518 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea9bedb5-ec22-4cad-bc24-a848426c62a4 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:21:28.315961 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea9bedb5-ec22-4cad-bc24-a848426c62a4 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:21:28.316377 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG oslo_concurrency.lockutils [None req-ea9bedb5-ec22-4cad-bc24-a848426c62a4 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121037) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:21:28.330059 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.wsgi [None req-ea9bedb5-ec22-4cad-bc24-a848426c62a4 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] HTTP exception thrown: Instance 134e1ff6-ab5a-41a8-9afb-8891869494bf could not be found. Sep 04 01:21:28.333846 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-ea9bedb5-ec22-4cad-bc24-a848426c62a4 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Request method failure captured: Sep 04 01:21:28.333846 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: request: DELETE /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf HTTP/1.1 Sep 04 01:21:28.333846 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept: application/json Sep 04 01:21:28.333846 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Accept-Encoding: identity Sep 04 01:21:28.333846 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Connection: close Sep 04 01:21:28.333846 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Length: 0 Sep 04 01:21:28.333846 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Content-Type: application/json Sep 04 01:21:28.333846 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Host: 10.222.0.42 Sep 04 01:21:28.333846 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: Openstack-System-Scope: None Sep 04 01:21:28.333846 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: User-Agent: python-urllib3/1.26.12 Sep 04 01:21:28.333846 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Auth-Token: gAAAAABjE_v3xFr83GF5HAJz0F1ePckFApHoM5uGMmIbR4HzFsfIZD_EnhL2OGYcBnvrJRXBe6apIZ4muzHym6lqQRHT8CWNei8sBt1BevsxBM3QKX99qXYG3ttnjXaWqmmKrkT5XoKcl347t24GOiR5KkDxLGu96o2UlmXDRT33-ZG56RntQ2w Sep 04 01:21:28.333846 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Id: None Sep 04 01:21:28.333846 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Domain-Name: None Sep 04 01:21:28.333846 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Identity-Status: Confirmed Sep 04 01:21:28.333846 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Is-Admin-Project: True Sep 04 01:21:28.333846 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Id: default Sep 04 01:21:28.333846 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Domain-Name: Default Sep 04 01:21:28.333846 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Id: 2506b96ac13541e6a7a47c5aedca67e5 Sep 04 01:21:28.333846 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Project-Name: tempest-TestVolumeBootPattern-1801805144 Sep 04 01:21:28.333846 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Role: reader,member Sep 04 01:21:28.333846 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Roles: reader,member Sep 04 01:21:28.336220 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2506b96ac13541e6a7a47c5aedca67e5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_2506b96ac13541e6a7a47c5aedca67e5", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2506b96ac13541e6a7a47c5aedca67e5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/2506b96ac13541e6a7a47c5aedca67e5"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:21:28.336220 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant: tempest-TestVolumeBootPattern-1801805144 Sep 04 01:21:28.336220 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Id: 2506b96ac13541e6a7a47c5aedca67e5 Sep 04 01:21:28.336220 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-Tenant-Name: tempest-TestVolumeBootPattern-1801805144 Sep 04 01:21:28.336220 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User: tempest-TestVolumeBootPattern-1801805144-project Sep 04 01:21:28.336220 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Id: default Sep 04 01:21:28.336220 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Domain-Name: Default Sep 04 01:21:28.336220 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Id: 48c4f0c78a364c00a793666eeb5a520c Sep 04 01:21:28.336220 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: X-User-Name: tempest-TestVolumeBootPattern-1801805144-project Sep 04 01:21:28.336220 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: method: > Sep 04 01:21:28.336220 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: exception: Instance 134e1ff6-ab5a-41a8-9afb-8891869494bf could not be found. Sep 04 01:21:28.336220 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: action_args: {'id': '134e1ff6-ab5a-41a8-9afb-8891869494bf'} Sep 04 01:21:28.337412 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:21:28.337412 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:21:28.337412 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:21:28.337412 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:21:28.337412 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:21:28.337412 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:21:28.337412 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:21:28.337412 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:21:28.337412 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:21:28.337412 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:21:28.337412 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 134e1ff6-ab5a-41a8-9afb-8891869494bf could not be found. Sep 04 01:21:28.337412 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:21:28.337412 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:21:28.337412 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:21:28.337412 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:21:28.337412 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:21:28.337412 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:21:28.337412 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:21:28.337412 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:21:28.337412 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:21:28.339173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:21:28.339173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 04 01:21:28.339173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 04 01:21:28.339173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 04 01:21:28.339173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 04 01:21:28.339173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:21:28.339173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:21:28.339173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:21:28.339173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:21:28.339173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 134e1ff6-ab5a-41a8-9afb-8891869494bf could not be found. Sep 04 01:21:28.339173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:21:28.339173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:21:28.339173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:21:28.339173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:21:28.339173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:21:28.339173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:21:28.339173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:21:28.339173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:21:28.339173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 134e1ff6-ab5a-41a8-9afb-8891869494bf could not be found. Sep 04 01:21:28.339173 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: ERROR nova.api.openstack.wsgi Sep 04 01:21:28.340816 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-ea9bedb5-ec22-4cad-bc24-a848426c62a4 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Returning 404 to user: Instance 134e1ff6-ab5a-41a8-9afb-8891869494bf could not be found. {{(pid=121037) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:21:28.340816 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-ea9bedb5-ec22-4cad-bc24-a848426c62a4 tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "DELETE /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf" status: 404 len: 111 microversion: 2.1 time: 0.029922 Sep 04 01:21:28.340816 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 632/1290] 10.222.0.42 () {58 vars in 1239 bytes} [Sun Sep 4 01:21:28 2022] DELETE /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 04 01:21:28.343209 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-33d38cb8-3862-4795-8a7e-2182b7e2290b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '>' {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:21:28.343817 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.compute.api [None req-33d38cb8-3862-4795-8a7e-2182b7e2290b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] [instance: 134e1ff6-ab5a-41a8-9afb-8891869494bf] Fetching instance by UUID {{(pid=121036) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 04 01:21:28.349675 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-33d38cb8-3862-4795-8a7e-2182b7e2290b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:21:28.350181 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-33d38cb8-3862-4795-8a7e-2182b7e2290b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:21:28.352626 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-33d38cb8-3862-4795-8a7e-2182b7e2290b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:21:28.352626 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-33d38cb8-3862-4795-8a7e-2182b7e2290b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Acquiring lock "cf96709b-43a7-4964-94cf-103a5b1a1016" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:385}} Sep 04 01:21:28.352626 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-33d38cb8-3862-4795-8a7e-2182b7e2290b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:390}} Sep 04 01:21:28.352626 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG oslo_concurrency.lockutils [None req-33d38cb8-3862-4795-8a7e-2182b7e2290b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Lock "cf96709b-43a7-4964-94cf-103a5b1a1016" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=121036) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 04 01:21:28.367988 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.wsgi [None req-33d38cb8-3862-4795-8a7e-2182b7e2290b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] HTTP exception thrown: Instance 134e1ff6-ab5a-41a8-9afb-8891869494bf could not be found. Sep 04 01:21:28.369517 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-33d38cb8-3862-4795-8a7e-2182b7e2290b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Request method failure captured: Sep 04 01:21:28.369517 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: request: GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf HTTP/1.1 Sep 04 01:21:28.369517 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept: application/json Sep 04 01:21:28.369517 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Accept-Encoding: identity Sep 04 01:21:28.369517 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Connection: close Sep 04 01:21:28.369517 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Length: 0 Sep 04 01:21:28.369517 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Content-Type: application/json Sep 04 01:21:28.369517 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Host: 10.222.0.42 Sep 04 01:21:28.369517 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: Openstack-System-Scope: None Sep 04 01:21:28.369517 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: User-Agent: python-urllib3/1.26.12 Sep 04 01:21:28.369517 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Auth-Token: gAAAAABjE_v3xFr83GF5HAJz0F1ePckFApHoM5uGMmIbR4HzFsfIZD_EnhL2OGYcBnvrJRXBe6apIZ4muzHym6lqQRHT8CWNei8sBt1BevsxBM3QKX99qXYG3ttnjXaWqmmKrkT5XoKcl347t24GOiR5KkDxLGu96o2UlmXDRT33-ZG56RntQ2w Sep 04 01:21:28.369517 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Id: None Sep 04 01:21:28.369517 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Domain-Name: None Sep 04 01:21:28.369517 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Identity-Status: Confirmed Sep 04 01:21:28.369517 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Is-Admin-Project: True Sep 04 01:21:28.369517 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Id: default Sep 04 01:21:28.369517 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Domain-Name: Default Sep 04 01:21:28.369517 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Id: 2506b96ac13541e6a7a47c5aedca67e5 Sep 04 01:21:28.369517 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Project-Name: tempest-TestVolumeBootPattern-1801805144 Sep 04 01:21:28.369517 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Role: reader,member Sep 04 01:21:28.369517 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Roles: reader,member Sep 04 01:21:28.371463 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Service-Catalog: [{"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2506b96ac13541e6a7a47c5aedca67e5"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:8080/v1/AUTH_2506b96ac13541e6a7a47c5aedca67e5", "adminURL": "http://10.222.0.42:8080"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42:9696/networking"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/volume/v3/2506b96ac13541e6a7a47c5aedca67e5"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2/2506b96ac13541e6a7a47c5aedca67e5"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/placement"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.42/compute/v2.1"}]}] Sep 04 01:21:28.371463 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant: tempest-TestVolumeBootPattern-1801805144 Sep 04 01:21:28.371463 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Id: 2506b96ac13541e6a7a47c5aedca67e5 Sep 04 01:21:28.371463 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-Tenant-Name: tempest-TestVolumeBootPattern-1801805144 Sep 04 01:21:28.371463 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User: tempest-TestVolumeBootPattern-1801805144-project Sep 04 01:21:28.371463 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Id: default Sep 04 01:21:28.371463 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Domain-Name: Default Sep 04 01:21:28.371463 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Id: 48c4f0c78a364c00a793666eeb5a520c Sep 04 01:21:28.371463 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: X-User-Name: tempest-TestVolumeBootPattern-1801805144-project Sep 04 01:21:28.371463 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: method: > Sep 04 01:21:28.371463 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: exception: Instance 134e1ff6-ab5a-41a8-9afb-8891869494bf could not be found. Sep 04 01:21:28.371463 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: action_args: {'id': '134e1ff6-ab5a-41a8-9afb-8891869494bf'} Sep 04 01:21:28.372551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: {{(pid=121036) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 04 01:21:28.372551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:21:28.372551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 04 01:21:28.372551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 04 01:21:28.372551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 04 01:21:28.372551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 04 01:21:28.372551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 04 01:21:28.372551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 04 01:21:28.372551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 04 01:21:28.372551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 04 01:21:28.372551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 134e1ff6-ab5a-41a8-9afb-8891869494bf could not be found. Sep 04 01:21:28.372551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:21:28.372551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:21:28.372551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:21:28.372551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:21:28.372551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:21:28.372551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:21:28.372551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 04 01:21:28.372551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 04 01:21:28.372551 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 04 01:21:28.374516 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 04 01:21:28.374516 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 04 01:21:28.374516 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 04 01:21:28.374516 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 04 01:21:28.374516 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 04 01:21:28.374516 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 04 01:21:28.374516 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 04 01:21:28.374516 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 134e1ff6-ab5a-41a8-9afb-8891869494bf could not be found. Sep 04 01:21:28.374516 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:21:28.374516 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 04 01:21:28.374516 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:21:28.374516 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 04 01:21:28.374516 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 04 01:21:28.374516 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 04 01:21:28.374516 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 04 01:21:28.374516 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 04 01:21:28.374516 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 134e1ff6-ab5a-41a8-9afb-8891869494bf could not be found. Sep 04 01:21:28.374516 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: ERROR nova.api.openstack.wsgi Sep 04 01:21:28.375988 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: DEBUG nova.api.openstack.wsgi [None req-33d38cb8-3862-4795-8a7e-2182b7e2290b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Returning 404 to user: Instance 134e1ff6-ab5a-41a8-9afb-8891869494bf could not be found. {{(pid=121036) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 04 01:21:28.375988 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: INFO nova.api.openstack.requestlog [None req-33d38cb8-3862-4795-8a7e-2182b7e2290b tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf" status: 404 len: 111 microversion: 2.1 time: 0.031984 Sep 04 01:21:28.375988 dv-cinder-master-3-iscsi devstack@n-api.service[121036]: [pid: 121036|app: 0|req: 659/1291] 10.222.0.42 () {58 vars in 1236 bytes} [Sun Sep 4 01:21:28 2022] GET /compute/v2.1/servers/134e1ff6-ab5a-41a8-9afb-8891869494bf => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 04 01:21:29.872514 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: DEBUG nova.api.openstack.wsgi [None req-8737ff29-7a8f-4e56-8675-a2935d9ef5ed tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] Calling method '.version_select at 0x7fc10c599ca0>' {{(pid=121037) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 04 01:21:29.968329 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: INFO nova.api.openstack.requestlog [None req-8737ff29-7a8f-4e56-8675-a2935d9ef5ed tempest-TestVolumeBootPattern-1801805144 tempest-TestVolumeBootPattern-1801805144-project] 10.222.0.42 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-436573754" status: 202 len: 0 microversion: 2.1 time: 0.099690 Sep 04 01:21:29.968937 dv-cinder-master-3-iscsi devstack@n-api.service[121037]: [pid: 121037|app: 0|req: 633/1292] 10.222.0.42 () {58 vars in 1260 bytes} [Sun Sep 4 01:21:29 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-436573754 => generated 0 bytes in 101 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0)